In this paper we propose a novel experimental design-based algorithm to minimize regret in online stochastic linear and combinatorial bandits. While existing literature tends to focus on optimism-based algorithms--which have been shown to be suboptimal in many cases--our approach carefully plans which action to take by balancing the tradeoff between information gain and reward, overcoming the failures of optimism. In addition, we leverage tools from the theory of suprema of empirical processes to obtain regret guarantees that scale with the Gaussian width of the action set, avoiding wasteful union bounds. We provide state-of-the-art finite time regret guarantees and show that our algorithm can be applied in both the bandit and semi-bandit feedback regime. In the combinatorial semi-bandit setting, we show that our algorithm is computationally efficient and relies only on calls to a linear maximization oracle. In addition, we show that with slight modification our algorithm can be used for pure exploration, obtaining state-of-the-art pure exploration guarantees in the semi-bandit setting. Finally, we provide, to the best of our knowledge, the first example where optimism fails in the semi-bandit regime, and show that in this setting our algorithm succeeds.
翻译:在本文中,我们提出一种新的实验性设计算法,以尽量减少在线随机线性和组合式强盗的遗憾。虽然现有文献倾向于侧重于基于乐观的算法,但在许多情况中,这些算法被证明是不最理想的。在组合半带式半带式设置中,我们表明我们的算法在计算上是有效的,只依赖于线性最大化或触摸。此外,我们通过对经验过程的想象性理论进行微小的修改,以获得规模的遗憾保证,使用高萨的动作宽度,避免浪费性结合界限。我们提供了最先进的有限时间保证,并表明我们的算法可以同时适用于土匪和半带状式反馈制度。在组合半带式半带式半带式设置中,我们表明我们的算法在计算上是有效的,只能依靠线性最大化或触觉的呼声。此外,我们证明我们的算法可以稍稍作修改后用于纯粹的探索,在半带式环境中获得最先进的纯度勘探保证。最后,我们为我们的知识提供了最佳的限定时间保证,并表明我们的知识可以同时同时应用。我们的第一个例子,即显示我们的乐观在半带式系统失败。