Evaluation of linear regression techniques for atmospheric applications: the importance of appropriate weighting
Abstract. Linear regression techniques are widely used in atmospheric science, but they are
often improperly applied due to lack of consideration or inappropriate
handling of measurement uncertainty. In this work, numerical experiments are
performed to evaluate the performance of five linear regression techniques,
significantly extending previous works by Chu and Saylor. The five techniques
are ordinary least squares (OLS), Deming regression (DR), orthogonal distance
regression (ODR), weighted ODR (WODR), and York regression (YR). We first
introduce a new data generation scheme that employs the Mersenne twister (MT)
pseudorandom number generator. The numerical simulations are also improved
by (a) refining the parameterization of nonlinear measurement
uncertainties, (b) inclusion of a linear measurement uncertainty, and (c) inclusion of WODR for comparison. Results show that DR, WODR and YR produce
an accurate slope, but the intercept by WODR and YR is overestimated and the
degree of bias is more pronounced with a low R2 XY dataset. The
importance of a properly weighting parameter λ in DR is investigated
by sensitivity tests, and it is found that an improper λ in DR can
lead to a bias in both the slope and intercept estimation. Because the
 λ calculation depends on the actual form of the measurement error,
it is essential to determine the exact form of measurement error in the XY 
data during the measurement stage. If a priori error in one of the variables
is unknown, or the measurement error described cannot be trusted, DR, WODR
and YR can provide the least biases in slope and intercept among all tested
regression techniques. For these reasons, DR, WODR and YR are recommended for
atmospheric studies when both X and Y data have measurement errors. An Igor
Pro-based program (Scatter Plot) was developed to facilitate the
implementation of error-in-variables regressions.