PILCO
14273320
PILCO: A Model Based and Data Efficient Approach to Policy Search
In this paper, we introduce PILCO, a practical, data efficient model based policy search method. PILCO reduces model bias, one of the key problems of model based reinforcement learning, in a principled way. By learning a probabilistic dynamics model and explicitly incorporating model uncertainty into long term planning, PILCO can cope with very little data and facilitates learning from scratch in only a few trials. Policy evaluation is performed in closed form using state of the art approximate inference. Furthermore, policy gradients are computed analytically for policy improvement. We report unprecedented learning efficiency on challenging and high dimensional control tasks.
ICML
Marc Peter Deisenroth Carl E Rasmussen
214264910
Parameter design and optimization of a flight attitude simulator system based on PILCO framework
Yang Ye Feng Yang Ye-feng Deng Kai Deng Kai Zuo Ying Qi Zuo Ying-qi Ban Xiao Jun Ban Xiao-jun Huang Xian Lin Huang Xian-lin
208103402
A study of the Combination of LQR Control and PILCO Reinforcement Learning
Journal of Institute of Control, Robotics and Systems
Jaehyun Yoo
17139307
Improving PILCO with Bayesian Neural Network Dynamics Models
Model based reinforcement learning (RL) allows an agent to discover good policies with a small number of trials by generalising observed transitions. Data efficiency can be further improved with a probabilistic model of the agent's ignorance about the world, allowing it to choose actions under uncertainty. Bayesian modelling offers tools for this task, with PILCO [1] being a prominent example, achieving state of theart data efficiency on low dimensional RL benchmarks. But PILCO relies on Gaussian processes (GPs) which prohibits its applicability to problems that require a larger number of trials to be solved. Further, PILCO does not consider temporal correlation in model uncertainty between successive state transitions, which results in PILCO underestimating state uncertainty at future time steps [2] In this paper we extend PILCO's framework to use Bayesian deep dynamics models with approximate variational inference, allowing PILCO to scale linearly with number of trials and observation space dimensionality. Using particle methods we sample dynamics function realisations, and obtain lower cumulative cost than PILCO. We give insights into the modelling assumptions made in PILCO, and show that moment matching is a crucial simplifying assumption made by the model. Our implementation can leverage GPU architectures, offering faster running time than PILCO, and will allow structured observation spaces to be modelled (images or higher dimensional inputs) in the future.
Rowan McAllister Carl E Rasmussen
30718861
Data Efficient Policy Search using PILCO and Directed Exploration
Reinforcement learning (RL) algorithms solve general sequential decision making problems through learning by trial and error. Many reinforcement learning algorithms are proven to find a good or optimal controller, but may take many interactions with the environment to do so. For real world tasks, this is often impractical, as letting a learner interact with the environment takes time and can be costly.
Rowan McAllister
171835919
Vodenje nestabilnega nelinearnega sistema z algoritmom PILCO
V diplomskem delu smo ovrednotili delovanje algoritma PILCO, metode za iskanje regulacijskega zakona, na nestabilnem nelinearnem sistemu tretjega reda s kvantizirano amplitudo izhodnega signala. Na zacetku so predstavljene teoreticne osnove algoritma, ki zajemajo Gaussove procese in spodbujevano ucenje. Sledi opis delovanja samega algoritma PILCO ter kratek primer uporabe. V tretjem poglavju je opisan matematicni model sistema. Sistem, o katerem govorimo, je hidravlicna naprava, kjer s crpalko spreminjamo velikost zracnega mehurcka v plovcu in tako vplivamo na njegovo globino v vodi. Vsi eksperimenti so bili izvedeni kot racunalniska simulacija modela naprave. Cilj eksperimenta je voditi sistem, tako da plovec doseze in vzdrzuje doloceno globino. V nadaljevanju sledijo opisi potekov eksperimentov. Najprej smo, da bi ugotovili ustrezen cas vzorcenja, izvedli vec eksperimentov, kjer je bila naprava vodena z regulatorjem PID. Za tem pa smo izvedli dva eksperimenta z algoritmom PILCO; prvic je bila amplituda izhodnega signala zvezna, drugic pa kvantizirana. Algoritem PILCO deluje najbolje pri vodenju sistema z zvezno amplitudo izhodnega signala. Pri sistemu s kvantizirano amplitudo izhodnega signala deluje slabse. Za iskanje uspesnega regulacijskega zakona potrebuje veliko vec iteracij, referencni vrednosti pa ne sledi vec tako natancno.
Gregor Mavec
217738902
Situacion Economica y Financiera de la Empresa de servicios de Rectificaciones y Torneria Pilco E.I.R.L. y su Influencia en la toma de decisiones en la ciudad de Juliaca, periodos 2012 2013
Mario Espinoza Quispe
194262664
Enrique Pilco Paz. Musiciens, religion et societe dans les Andes au XXe siecle (Perou) Des voix dans la penombre
Este primer libro de Enrique Pilco, publicado poco despues de la sustentacion de su tesis doctoral en Francia (titulada Des voix dans la penombre. Le catholicisme cuzquenien a travers les hymnes religieux en Quechua. Musique, religion et societe dans les Andes du XXe siecle) cuenta con un prefacio de Carmen Bernand, su directora de tesis. En su presentacion, advierte que Pilco que <es ante todo un musico, un artista> (p. 11) no pretende aqui <batirse con los historiadores y musicologos del.
Rivera Andia Juan Javier
161078310
Enrique PILCO PAZ Musiciens, religion et societe dans les Andes au XXe siecle (Perou) Des voix dans la penombre. Preface de Carmen Bernand. Paris L'Harmattan
Issu d'une these soutenue en 2010 a l'EHESS sous la direction de Carmen Bernand, cet ouvrage nous plonge dans le monde des societes traditionnelles villageoises andines, plus precisement celles de la region de Cuzco autour de la musique associee a la liturgie et chantee en langue quechua. En effet, contrairement a ce qui se pratiquait dans les enceintes religieuses telles que les seminaires et les couvents, la catechisation des Indiens se fit dans les paroisses la plupart du temps en langue
Michel Plisson
61258558
PILCO Code Documentation v0.9
We describe a Matlab package for the pilco policy search framework for data efficient reinforcement learning. This package implements the pilco learning framework in multiple different scenarios with continuous states and actions: pendulum swing up, cart pole swing up, double pendulum swing up (with either one or two actuators) cart double pendulum swing up, and unicycling. Results from some of these scenarios have been presented previously in [3, 4] The high level steps of the pilco algorithm, which are also implemented in this software package, are the following: Learn a Gaussian process (GP) model of the system dynamics, perform deterministic approximate inference for policy evaluation, update the policy parameters using exact gradient information, apply the learned controller to the system. The software package provides an interface that allows for setting up novel tasks without the need to be familiar with the intricate details of model learning, policy evaluation and improvement.
Marc Peter Deisenroth Andrew McHutchon