The Square Root Rule for Adaptive Importance Sampling
In adaptive importance sampling and other contexts, we have K > 1 unbiased and uncorrelated estimates μ^k of a common quantity μ. The optimal unbiased linear combination weights them inversely to their variances, but those weights are unknown and hard to estimate. A simple deterministic square root rule based on a working model that Var(μ^k) ∝ k−1/2 gives an unbiased estimate of μ that is nearly optimal under a wide range of alternative variance patterns. We show that if Var(μ^k)∝ k−y for an unknown rate parameter y∈[0,1], then the square root rule yields the optimal variance rate with a constant that is too large by at most 9/8 for any 0 ⩽ y⩽ 1 and any number K of estimates. Numerical work shows that rule is similarly robust to some other patterns with mildly decreasing variance as k increases.