text: Confidence interval | Project Gutenberg Self-Publishing - eBooks | Read eBooks online
  Article Id: WHEBN0000280911
  Subject: Statistical inference, Student's t-distribution, Estimation statistics, Group size measures, In vitro fertilisation
  Collection: Biostatistics, Econometrics, Market Research, Measurement, Psephology, Statistical Inference, Statistical Intervals, Statistical Terminology
  In statistics, a confidence interval (CI) is a type of interval estimate of a population parameter. It is an observed interval (i.e., it is calculated from the observations), in principle different from sample to sample, that frequently includes the value of an unobservable parameter of interest if the experiment is repeated. How frequently the observed interval contains the parameter is determined by the confidence level or confidence coefficient. More specifically, the meaning of the term "confidence level" is that, if CI are constructed across many separate data analyses of replicated (and possibly different) experiments, the proportion of such intervals that contain the true value of the parameter will match the given confidence level.[1][2][3] Whereas two-sided confidence limits form a confidence interval, their one-sided counterparts are referred to as lower or upper confidence bounds.
  Confidence intervals consist of a range of values (interval) that act as good estimates of the unknown population parameter; however, none of these values may cover the value of the parameter. It does not describe any single sample. This value is represented by a percentage, so when we say, "we are 99% confident that the true value of the parameter is in our confidence interval", we express that 99% of the hypothetically observed confidence intervals will hold the true value of the parameter. After any particular sample is taken, the population parameter is either in the interval realized or not; it is not a matter of chance. The desired level of confidence is set by the researcher (not determined by data). If a corresponding hypothesis test is performed, the confidence level is the complement of respective level of significance, i.e. a 95% confidence interval reflects a significance level of 0.05.[4] The confidence interval contains the parameter values that, when tested, should not be rejected with the same sample. Greater levels of variance yield larger confidence intervals, and hence less precise estimates of the parameter. Confidence intervals of difference parameters not containing 0 imply that there is a statistically significant difference between the populations.
  Conceptual basis 1
  Meaning and interpretation 1.2
  Misunderstandings 1.2.1
  Philosophical issues 1.3
  Relationship with other statistical topics 1.4
  Statistical hypothesis testing 1.4.1
  Confidence region 1.4.2
  Confidence band 1.4.3
  Approximate confidence intervals 2.1.1
  Desirable properties 2.2
  Methods of derivation 2.3
  Interpretation 3.1.1
  Theoretical example 3.2
  Alternatives and critiques 4
  Comparison to prediction intervals 4.1
  Comparison to tolerance intervals 4.2
  Comparison to Bayesian interval estimates 4.3
  Confidence intervals for proportions and related quantities 4.4
  Confidence interval for specific distributions 5.1
  For example, a confidence interval can be used to describe how reliable survey results are. In a poll of election voting-intentions, the result might be that 40% of respondents intend to vote for a certain party. A 99% confidence interval for the proportion in the whole population having the same intention on the survey might be 30% to 50%. From the same data one may calculate a 90% confidence interval, which in this case might be 37% to 43%. A major factor determining the length of a confidence interval is the size of the sample used in the estimation procedure, for example the number of people taking part in a survey.
  The confidence interval can be expressed in terms of samples (or repeated samples): "Were this procedure to be repeated on multiple samples, the calculated confidence interval (which would differ for each sample) would encompass the true population parameter 90% of the time."[1] Note that this does not refer to repeated measurement of the same sample, but repeated sampling.[2]
  The confidence interval can be expressed in terms of a single sample: "There is a 90% probability that the calculated confidence interval from some future experiment encompasses the true value of the population parameter." Note this is a probability statement about the confidence interval, not the population parameter. This considers the probability associated with a confidence interval from a pre-experiment point of view, in the same context in which arguments for the random allocation of treatments to study items are made. Here the experimenter sets out the way in which they intend to calculate a confidence interval and know, before they do the actual experiment, that the interval they will end up calculating has a certain chance of covering the true but unknown value.[3] This is very similar to the "repeated sample" interpretation above, except that it avoids relying on considering hypothetical repeats of a sampling procedure that may not be repeatable in any meaningful sense. See Neyman construction.
  A 95% confidence interval does not mean that for a given realised interval calculated from sample data there is a 95% probability the population parameter lies within the interval, nor that there is a 95% probability that the interval covers the population parameter.[11] Once an experiment is done and an interval calculated, this interval either covers the parameter value or it does not, it is no longer a matter of probability. The 95% probability relates to the reliability of the estimation procedure, not to a specific calculated interval.[12] Neyman himself (the original proponent of confidence intervals) made this point in his original paper:[3]
  "It must be stressed, however, that having seen the value [of the data], Neyman-Pearson theory never permits one to conclude that the specific confidence interval formed covers the true value of 0 with either (1 - α)100% probability or (1 - α)100% degree of confidence. Seidenfeld's remark seems rooted in a (not uncommon) desire for Neyman-Pearson confidence intervals to provide something which they cannot legitimately provide; namely, a measure of the degree of probability, belief, or support that an unknown parameter value lies in a specific interval. Following Savage (1962), the probability that a parameter lies in a specific interval may be referred to as a measure of final precision. While a measure of final precision may seem desirable, and while confidence levels are often (wrongly) interpreted as providing such a measure, no such interpretation is warranted. Admittedly, such a misinterpretation is encouraged by the word 'confidence'."
  A confidence interval is not a range of plausible values for the sample mean, though it may be understood as an estimate of plausible values for the population parameter.
  A particular confidence interval of 95% calculated from an experiment does not mean that there is a 95% probability of a sample mean from a repeat of the experiment falling within this interval.[14]
  The principle behind confidence intervals was formulated to provide an answer to the question raised in statistical inference of how to deal with the uncertainty inherent in results derived from data that are themselves only a randomly selected subset of a population. There are other answers, notably that provided by Bayesian inference in the form of credible intervals. Confidence intervals correspond to a chosen rule for determining the confidence bounds, where this rule is essentially determined before any data are obtained, or before an experiment is done. The rule is defined such that over all possible datasets that might be obtained, there is a high probability ("high" is specifically quantified) that the interval determined by the rule will include the true value of the quantity under consideration. That is a fairly straightforward and reasonable way of specifying a rule for determining uncertainty intervals. The Bayesian approach appears to offer intervals that can, subject to acceptance of an interpretation of "probability" as Bayesian probability, be interpreted as meaning that the specific interval calculated from a given dataset has a certain probability of including the true value, conditional on the data and other information available. The confidence interval approach does not allow this, since in this formulation and at this same stage, both the bounds of the interval and the true values are fixed values and there is no randomness involved.
  For example, in the poll example outlined in the introduction, to be 95% confident that the actual number of voters intending to vote for the party in question is between 36% and 44%, should not be interpreted in the common-sense interpretation that there is a 95% probability that the actual number of voters intending to vote for the party in question is between 36% and 44%. The actual meaning of confidence levels and confidence intervals is rather more subtle. In the above case, a correct interpretation would be as follows: If the polling were repeated a large number of times (you could produce a 95% confidence interval for your polling confidence interval), each time generating about a 95% confidence interval from the poll sample, then 95% of the generated intervals would contain the true percentage of voters who intend to vote for the given party. Each time the polling is repeated, a different confidence interval is produced; hence, it is not possible to make absolute statements about probabilities for any one given interval. For more information, see the section on meaning and interpretation.
  The questions concerning how an interval expressing uncertainty in an estimate might be formulated, and of how such intervals might be interpreted, are not strictly mathematical problems and are philosophically problematic.[15] Mathematics can take over once the basic principles of an approach to 'inference' have been established, but it has only a limited role in saying why one approach should be preferred to another: For example, a confidence level of 95% is often used in the biological sciences, but this is a matter of convention or arbitration. In the physical sciences, a much higher level may be used.[16]
  More generally, given the availability of a hypothesis testing procedure that can test the null hypothesis θ = θ0 against the alternative that θ ≠ θ0 for any value of θ0, then a confidence interval with confidence level γ = 1 − α can be defined as containing any number θ0 for which the corresponding null hypothesis is not rejected at significance level α.[17]
  If the estimates of two parameters (for example, the mean values of a variable in two independent groups) have confidence intervals that do not overlap, then the difference between the two values is more significant than indicated by the individual values of α.[18] So, this "test" is too conservative and can lead to a result that is more significant than the individual values of α would indicate. If two confidence intervals overlap, the two means still may be significantly different.[19][20][21] Accordingly, and consistent with the Mantel-Haenszel Chi-squared test, is a proposed fix whereby one reduces the error bounds for the two means by multiplying them by the square root of 1/2 (0.707107) before making the comparison.[22]
  Confidence bands are closely related to confidence intervals, which represent the uncertainty in an estimate of a single numerical value. "As confidence intervals, by construction, only refer to a single point, they are narrower (at this point) than a confidence band which is supposed to hold simultaneously at many points."[23]
  {\Pr}_{\theta,\phi}(u(X)<\theta
  to an acceptable level of approximation. Alternatively, some authors[24] simply require that
  Invariance. In many applications the quantity being estimated might not be tightly defined as such. For example, a survey might result in an estimate of the median income in a population, but it might equally be considered as providing an estimate of the logarithm of the median income, given that this is a common scale for presenting graphical results. It would be desirable that the method used for constructing a confidence interval for the median income would give equivalent results when applied to constructing a confidence interval for the logarithm of the median income: specifically the values at the ends of the latter interval would be the logarithms of the values at the ends of former interval..
  Where estimates are constructed using the maximum likelihood principle, the theory for this provides two ways of constructing confidence intervals or confidence regions for the estimates. One way is by using Wilks's theorem to find all the possible values of \theta that fulfill the following restriction:[25]
  \ln \left( L\left( \theta \right) \right)\ge \ln \left( L\left( {\hat{\theta }} \right) \right)-\frac{1}{2}_{1,1-\alpha }
  If significance tests are available for general values of a parameter, then confidence intervals/regions can be constructed by including in the 100p% confidence region all those points for which the significance test of the null hypothesis that the true value is the given value is not rejected at a significance level of (1-p).[17]
  A machine fills cups with a liquid, and is supposed to be adjusted so that the content of the cups is 250 g of liquid. As the machine cannot fill every cup with exactly 250.0 g, the content added to individual cups shows some variation, and is considered a random variable X. This variation is assumed to be normally distributed (although this assumption is not necessary for the theory to work) around the desired average of 250 g, with a standard deviation, σ, of 2.5 g. To determine if the machine is adequately calibrated, a sample of n = 25 cups of liquid are chosen at random and the cups are weighed. The resulting measured masses of liquid are X1, ..., X25, a random sample from X.
  \hat \mu=\bar X = \frac{1}{n}\sum_{i=1}^n X_i.
  \bar x=\frac {1}{25} \sum_{i=1}^{25} x_i = 250.2\,\text{grams}.
  \frac {\sigma}{\sqrt{n}}=\frac {2.5~\text{g}}{\sqrt{25}}=0.5\ \text{grams}
  Z = \frac {\bar X-\mu}{\sigma/\sqrt{n}} =\frac {\bar X-\mu}{0.5}
  \!P(-z\le Z\le z) = 1-\alpha = 0.95.
  \begin{align} \Phi(z) & = P(Z \le z) = 1 - \tfrac{\alpha}{2} = 0.975,\\[6pt] z & = \Phi^{-1}(\Phi(z)) = \Phi^{-1}(0.975) = 1.96, \end{align}
  \begin{align} 0.95 & = 1-\alpha=P(-z \le Z \le z)=P \left(-1.96 \le \frac {\bar X-\mu}{\sigma/\sqrt{n}} \le 1.96 \right) \\[6pt] & = P \left( \bar X - 1.96 \frac{\sigma}{\sqrt{n}} \le \mu \le \bar X + 1.96 \frac{\sigma}{\sqrt{n}}\right) \end{align}.
  \text{Lower endpoint} = \bar X - 1.96 \frac{\sigma}{\sqrt{n}},
  \text{Upper endpoint} = \bar X + 1.96 \frac{\sigma}{\sqrt{n}}.
  \begin{align} 0.95 & = P\left(\bar X - 1.96 \times 0.5 \le \mu \le \bar X + 1.96 \times 0.5\right) \\[6pt] & = P \left( \bar X - 0.98 \le \mu \le \bar X + 0.98 \right). \end{align}
  As the standard deviation of the population σ is known in this case, the distribution of the sample mean \bar X is a normal distribution with \mu the only unknown parameter. In the theoretical example below, the parameter σ is also unknown, which calls for using the Student's t-distribution.
  \! \bar X - 0{.}98
  \! \bar X + 0.98.
  (\bar{x}-0.98,\, \bar{x}+0.98).
  (\bar x - 0.98;\bar x + 0.98) = (250.2 - 0.98; 250.2 + 0.98) = (249.22; 251.18).\,
  \bar{X}=(X_1+\cdots+X_n)/n\,,
  S^2=\frac{1}{n-1}\sum_{i=1}^n\left(X_i-\bar{X}\,\right)^2.
  T=\frac{\bar{X}-\mu}{S/\sqrt{n}}
  has a Student's t-distribution with n − 1 degrees of freedom.[26] Note that the distribution of T does not depend on the values of the unobservable parameters μ and σ2; i.e., it is a pivotal quantity. Suppose we wanted to calculate a 95% confidence interval for μ. Then, denoting c as the 97.5th percentile of this distribution,
  \Pr\left(-c\le T \le c\right)=0.95\,
  \Pr\left(\bar{X} - \frac{cS}{\sqrt{n}} \le \mu \le \bar{X} + \frac{cS}{\sqrt{n}} \right)=0.95\,
  \left[ \bar{x} - \frac{cs}{\sqrt{n}}, \bar{x} + \frac{cs}{\sqrt{n}} \right], \,
  {\Pr}_{\theta,\phi}(u(X) < Y < v(X)) = \gamma\text{ for all }(\theta,\phi).\,
  A Bayesian interval estimate is called a credible interval. Using much of the same notation as above, the definition of a credible interval for the unknown true value of θ is, for a given γ,[27]
  \Pr(u(x)<\Theta
  Users of Bayesian methods, if they produced an interval estimate, would in contrast to confidence intervals, want to say "My degree of belief that the parameter is in fact in this interval is 90%,"[28] while users of prediction intervals would instead say "I predict that the next sample will fall in this interval 90% of the time."
  ^ a b Kendall, M.G. and Stuart, D.G. (1973) The Advanced Theory of Statistics. Vol 2: Inference and Relationship, Griffin, London. Section 20.4
  ^ Zar, J.H. (1984) Biostatistical Analysis. Prentice Hall International, New Jersey. pp 43–45
  ^ Hoekstra, R., R. D. Morey, J. N. Rouder, and E-J. Wagenmakers, 2014. Robust misinterpretation of confidence intervals. Psychonomic Bulletin Review, in press. [7]
  ^ Morey, R.D., Hoekstra, R., Lee, M.D., Rouder, J.N., Wagenmakers, E-J. (2015) The Fallacy of Placing Confidence in Confidence Intervals. in press. [8]
  ^ Pav Kalinowski. Understanding Confidence Intervals (CIs) and Effect Size Estimation. Association for Psychological Science Observer April 10, 2010. http://www.psychologicalscience.org/publications/observer/2010/april-10/understanding-confidence-intervals-cis-and-effect-size-estimation.html
  ^ Goldstein, H.; Healey, M.J.R. (1995). "The graphical presentation of a collection of means". Journal of the Royal Statistical Society 158: 175–77.
  ^ Wolfe R, Hanley J (Jan 2002). "If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2". CMAJ 166 (1): 65–6.
  ^ p.65 in W. Härdle, M. Müller, S. Sperlich, A. Werwatz (2004), Nonparametric and Semiparametric Models, Springer, ISBN 3540207228
  ^ Abramovich, Felix, and Ya'acov Ritov. Statistical Theory: A Concise Introduction. CRC Press, 2013. Pages 121-122
  ^ Bernardo JE, Smith, Adrian (2000). Bayesian theory. New York: Wiley. p. 259.
  ^ Cox D.R., Hinkley D.V. (1974) Theoretical Statistics, Chapman & Hall, p390
  Robinson, G.K. (1975). "Some Counterexamples to the Theory of Confidence Intervals". Biometrika 62: 155–161.