reinforcement learning portfolio allocation cryptocurrency optimal agent
58007060
Multi Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
This work demonstrates the potential of deep reinforcement learning techniques for transmit power control in wireless networks. Existing techniques typically find near optimal power allocations by solving a challenging optimization problem. Most of these algorithms are not scalable to large networks in real world scenarios because of their computational complexity and instantaneous cross cell channel state information (CSI) requirement. In this paper, a distributively executed dynamic power allocation scheme is developed based on model free deep reinforcement learning. Each transmitter collects CSI and quality of service (QoS) information from several neighbors and adapts its own transmit power accordingly. The objective is to maximize a weighted sum rate utility function, which can be particularized to achieve maximum sum rate or proportionally fair scheduling. Both random variations and delays in the CSI are inherently addressed using deep {Q} learning. For a typical network architecture, the proposed algorithm is shown to achieve near optimal power allocation in real time based on delayed CSI measurements available to the agents. The proposed scheme is especially suitable for practical scenarios where the system model is inaccurate and CSI delay is non negligible.
IEEE Journal on Selected Areas in Communications
Yasar Sinan Nasir Dongning Guo
195776345
Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation
Portfolio allocation is crucial for investment companies. However, getting the best strategy in a complex and dynamic stock market is challenging. In this paper, we propose a novel Adaptive Deep Deterministic Reinforcement Learning scheme (Adaptive DDPG) for the portfolio allocation task, which incorporates optimistic or pessimistic deep reinforcement learning that is reflected in the influence from prediction errors. Dow Jones 30 component stocks are selected as our trading stocks and their daily prices are used as the training and testing data. We train the Adaptive DDPG agent and obtain a trading strategy. The Adaptive DDPG's performance is compared with the vanilla DDPG, Dow Jones Industrial Average index and the traditional min variance and mean variance portfolio allocation strategies. Adaptive DDPG outperforms the baselines in terms of the investment return and the Sharpe ratio.
Xinyi Li Yinchuan Li Yuancheng Zhan Xiao-Yang Liu
202540651
Automatic Financial Trading Agent for Low risk Portfolio Management using Deep Reinforcement Learning
The autonomous trading agent is one of the most actively studied areas of artificial intelligence to solve the capital market portfolio management problem. The two primary goals of the portfolio management problem are maximizing profit and restrainting risk. However, most approaches to this problem solely take account of maximizing returns. Therefore, this paper proposes a deep reinforcement learning based trading agent that can manage the portfolio considering not only profit maximization but also risk restraint. We also propose a new target policy to allow the trading agent to learn to prefer low risk actions. The new target policy can be reflected in the update by adjusting the greediness for the optimal action through the hyper parameter. The proposed trading agent verifies the performance through the data of the cryptocurrency market. The Cryptocurrency market is the best test ground for testing our trading agents because of the huge amount of data accumulated every minute and the market volatility is extremely large. As a experimental result, during the test period, our agents achieved a return of 1800% and provided the least risky investment strategy among the existing methods. And, another experiment shows that the agent can maintain robust generalized performance even if market volatility is large or training period is short.
ArXiv
Wonsup Shin Seok-Jun Bu Sung-Bae Cho
29166962
Deep Reinforcement Learning Based Resource Allocation for V2V Communications
In this paper, we develop a novel decentralized resource allocation mechanism for vehicle to vehicle (V2V) communications based on deep reinforcement learning, which can be applied to both unicast and broadcast scenarios. According to the decentralized resource allocation mechanism, an autonomous "agent," a V2V link or a vehicle, makes its decisions to find the optimal sub band and power level for transmission without requiring or having to wait for global information. Since the proposed method is decentralized, it incurs only limited transmission overhead. From the simulation results, each agent can effectively learn to satisfy the stringent latency constraints on V2V links while minimizing the interference to vehicle to infrastructure communications.
IEEE Transactions on Vehicular Technology
Hao Ye Geoffrey Y Li Biing-Hwang Juang
86836682
Resource Allocation for Edge Computing in IoT Networks via Reinforcement Learning
In this paper, we consider resource allocation for edge computing in internet of things (IoT) networks. Specifically, each end device is considered as an agent, which makes its decisions on whether offloading the computation tasks to the edge devices or not. To minimize the long term weighted sum cost which includes the power consumption and the task execution latency, we consider the channel conditions between the end devices and the gateway, the computation task queue as well as the remaining computation resource of the end devices as the network states. The problem of making a series of decisions at the end devices is modelled as a Markov decision process and solved by the reinforcement learning approach. Therefore, we propose a near optimal task offloading algorithm based on greedy Q learning. Simulations validate the feasibility of our proposed algorithm, which achieves a better trade off between the power consumption and the task execution latency compared to these of edge computing and local computing modes.
ICC 2019 2019 IEEE International Conference on Communications (ICC)
Xiaolan Liu Zhijin Qin Yue Gao
202139240
Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks
Heterogeneous cellular networks can offload the mobile traffic and reduce the deployment costs, which have been considered to be a promising technique in the next generation wireless network. Due to the non convex and combinatorial characteristics, it is challenging to obtain an optimal strategy for the joint user association and resource allocation issue. In this paper, a reinforcement learning (RL) approach is proposed to achieve the maximum long term overall network utility while guaranteeing the quality of service requirements of user equipments (UEs) in the downlink of heterogeneous cellular networks. A distributed optimization method based on multi agent RL is developed. Moreover, to solve the computationally expensive problem with the large action space, multi agent deep RL method is proposed. Specifically, the state, action and reward function are defined for UEs, and dueling double deep Q network (D3QN) strategy is introduced to obtain the nearly optimal policy. Through message passing, the distributed UEs can obtain the global state space with a small communication overhead. With the double Q strategy and dueling architecture, D3QN can rapidly converge to a subgame perfect Nash equilibrium. Simulation results demonstrate that D3QN achieves the better performance than other RL approaches in solving large scale learning problems.
IEEE Transactions on Wireless Communications
Nan Zhao Ying-Chang Liang Dusit Tao Niyato Yiyang Pei Minghu Wu Yunhao Jiang
53291853
Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi Agent Reinforcement Learning
In this paper, we develop a distributed mechanism for spectrum sharing among a network of unmanned aerial vehicles (UAV) and licensed terrestrial networks. This method can provide a practical solution for situations where the UAV network may need external spectrum when dealing with congested spectrum or need to change its operational frequency due to security threats. Here we study a scenario where the UAV network performs a remote sensing mission. In this model, the UAVs are categorized to two clusters of relaying and sensing UAVs. The relay UAVs provide a relaying service for a licensed network to obtain spectrum access for the rest of UAVs that perform the sensing task. We develop a distributed mechanism in which the UAVs locally decide whether they need to participate in relaying or sensing considering the fact that communications among UAVs may not be feasible or reliable. The UAVs learn the optimal task allocation using a distributed reinforcement learning algorithm. Convergence of the algorithm is discussed and simulation results are presented for different scenarios to verify the convergence.
2019 16th IEEE Annual Consumer Communications Networking Conference (CCNC)
Alireza Shamsoshoara Mehrdad Khaledi Fatemeh Afghah Abolfazl Razi Jonathan D Ashdown
212778172
Multi agent Reinforcement Learning for Joint Wireless and Computational Resource Allocation in Mobile Edge Computing System
Mobile edge computing (MEC) is a new paradigm to provide computing capabilities at the edge of pervasive radio access networks in close proximity to intelligent terminals. In this paper, a resource allocation strategy based on the variable learning rate multi agent reinforcement learning (VLR MARL) algorithm is proposed in the MEC system to maximize the long term utility of all intelligent terminals while ensuring the intelligent terminals' quality of service requirement. The novelty of this algorithm is that each agent only needs to maintain its own action value function so that the computationally expensive issue with the large action space can be avoided. Moreover, the learning rate is changed according to the expected payoff of the current strategy to speed up convergence and get the optimal solution. Simulation results show our algorithm performs better than other reinforcement learning algorithm both on the learning speed and users' long term utilities.
ADHOCNETS
Yawen Zhang Weiwei Xia Feng Yan Huaqing Cheng Lianfeng Shen
145868358
A constrained portfolio trading system using particle swarm algorithm and recurrent reinforcement learning
Abstract This study extends a recurrent reinforcement portfolio allocation and rebalancing management system with complex portfolio constraints using particle swarm algorithms. In particular, we propose to use a combination of recurrent reinforcement learning (RRL) and particle swarm algorithm (PSO) with Calmar ratio for both asset allocation and constraint optimization. Using S&P100 index stocks, we show such a system with a Calmar ratio based objective function yields a better efficient frontier than the Sharpe ratio and mean variance based portfolios. By comparing with multiple PSO based long only constrained portfolios, we propose an optimal portfolio trading system that is capable of generating both long and short signals and handling the common portfolio constraints. We further develop an adaptive RRL PSO portfolio rebalancing decision system with a market condition stop loss retraining mechanism, and we show that the proposed portfolio trading system outperforms the benchmarks consistently especially under high transaction cost conditions.
Expert Syst. Appl.
Saud Almahdi Steve Y Yang
198967834
Large Scale Continuous Time Mean Variance Portfolio Allocation via Reinforcement Learning
We propose to solve large scale Markowitz mean variance (MV) portfolio allocation problem using reinforcement learning (RL) By adopting the recently developed continuous time exploratory control framework, we formulate the exploratory MV problem in high dimensions. We further show the optimality of a multivariate Gaussian feedback policy, with time decaying variance, in trading off exploration and exploitation. Based on a provable policy improvement theorem, we devise a scalable and data efficient RL algorithm and conduct large scale empirical tests using data from the S&P 500 stocks. We found that our method consistently achieves over 10% annualized returns and it outperforms econometric methods and the deep RL method by large margins, for both long and medium terms of investment with monthly and daily trading.
ArXiv
Haoran Wang Xun Yu Zhou