MDP Playground: A Design and Debug Testbed for Reinforcement Learning
We present MDP Playground, an efficient testbed for Reinforcement Learning 1 (RL) agents with orthogonal dimensions that can be controlled independently 2 to challenge agents in different ways and obtain varying degrees of hardness in 3 generated environments. We consider and allow control over a wide variety of 4 dimensions, including delayed rewards, rewardable sequences, density of rewards, 5 stochasticity, image representations, irrelevant features, time unit, action range 6 and more. We define a parameterised collection of fast-to-run toy environments 7 in OpenAI Gym by varying these dimensions and propose to use these for the 8 initial design and development of agents. We also provide wrappers that inject 9 these dimensions into complex environments from Atari and Mujoco to allow for 10 evaluating agent robustness. We further provide various example use-cases and 11 instructions on how to use MDP Playground to design and debug agents. We 12 believe that MDP Playground is a valuable testbed for researchers designing new, 13 adaptive and intelligent RL agents and those wanting to unit test their agents. 14