Decentralized Likelihood Quantile Networks for Improving Performance in Deep Multi-Agent Reinforcement Learning
Recent value-based multi-agent deep reinforcement learning methods employ optimism by limiting underestimation updates of the value function estimator through a carefully controlled learning rate (Omidshafiei et al., 2017) or a reduced update probability (Palmer et al., 2018). This value overestimation is meant to counteract negative effects caused by sub-optimal (but unobservable) teammate policies and exploration. This paper introduces a decentralized quantile estimator, which aims to improve performance through automatic scheduling. Our experiments show the method is more stable, sample efficient and more likely to converge to a joint optimal policy than previous methods.