Double Happiness: Enhancing the Coupled Gains of L-lag Coupling via Control Variates

The recently proposed L-lag coupling for unbiased MCMC (Biswas et al., 2019; Jacob et al., 2020) calls for a joint celebration by MCMC practitioners and theoreticians. For practitioners, it circumvents the thorny issue of deciding the burn-in period or when to terminate an MCMC sampling process, and opens the door for safe parallel implementation. For theoreticians, it provides a powerful tool to establish elegant and easily estimable bounds on the exact error of MCMC approximation at any finite number of iterates. A serendipitous observation about the bias correcting term leads us to introduce naturally available control variates into the L-lag coupling estimators. In turn, this extension enhances the coupled gains of L-lag coupling, because it results in more efficient unbiased estimators as well as a better bound on the total variation error of MCMC iterations, albeit the gains diminish with the numerical value of L. Specifically, the new bound is theoretically guaranteed to never exceed the one given previously. We also argue that L-lag coupling represents a long-sought-after coupling for the future, breaking a logjam of the coupling-from-the-past type of perfect sampling, by reducing the generally un-achievable requirement of being perfect to being unbiased, a worthwhile trade-off for ease of implementation in most practical situations. The theoretical analysis is supported by numerical experiments that show tighter bounds and a gain in efficiency when control variates are introduced.