Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Modern applications of the theory of Markov Decision Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because actual time span of these problems can be months or years, during which discounting factors due to e.g. interest rates are of practical concern. ,  show that, for such MDPs with discount rate α close to 1, under a common ergodicity assumption, a ighted difference beten two successive value function estimates obtained from the classical value iteration (VI) is a better approximation than the value function obtained directly from VI. Rigorous error bounds are established which in turn show that the approximation converges to the actual value function in a rate (αβ) k  with β < 1. This indicates a geometric convergence even if discount factor α → 1. Furthermore,  explicitly link the convergence speed to the system behaviors of the MDP using the notion of e-mixing time and extend our result to Q-functions. Numerical experiments are conducted to demonstrate the convergence properties of the proposed approximation scheme.
['Yinlam Chow', 'Junjie Qin']
I am writing a research paper on a idea Weighted difference approximation of value functions for slow-discounting Markov Decision Processes
[{"from": "human", "value": "I am writing a research paper on a idea Weighted difference approximation of value functions for slow-discounting Markov Decision Processes"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Modern applications of the theory of Markov Decision Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because actual time span of these problems can be months or years, during which discounting factors due to e.g. interest rates are of practical concern. ,  show that, for such MDPs with discount rate \u03b1 close to 1, under a common ergodicity assumption, a ighted difference beten two successive value function estimates obtained from the classical value iteration (VI) is a better approximation than the value function obtained directly from VI. Rigorous error bounds are established which in turn show that the approximation converges to the actual value function in a rate (\u03b1\u03b2) k  with \u03b2 < 1. This indicates a geometric convergence even if discount factor \u03b1 \u2192 1. Furthermore,  explicitly link the convergence speed to the system behaviors of the MDP using the notion of e-mixing time and extend our result to Q-functions. Numerical experiments are conducted to demonstrate the convergence properties of the proposed approximation scheme."}]