Restless bandits model the exploration vs. exploitation trade-off in a changing (non-stationary) world. Restless#R##N##R##N#bandits have been studied in both the context of continuously-changing (drifting) and change-point (sudden)#R##N##R##N#restlessness. In this work, we study specific classes of drifting restless bandits selected for their relevance to#R##N##R##N#modelling an online website optimization process. The contribution in this work is a simple, feasible weighted#R##N##R##N#least squares technique capable of utilizing contextual arm parameters while considering the parameter space#R##N##R##N#drifting non-stationary within reasonable bounds. We produce a reference implementation, then evaluate and#R##N##R##N#compare its performance in several different true world states, finding experimentally that performance is#R##N##R##N#robust to time drifting factors similar to those seen in many real world cases.
['Giuseppe Burtini', 'Jason L. Loeppky', 'Ramon Lawrence']
Improving Online Marketing Experiments with Drifting Multi-armed Bandits