Distribution & Critical Region
R Code
Step-by-Step
Quick Reference Table
The critical value is the number of standard errors from the center of the sampling distribution needed to capture the desired confidence level. It is the multiplier \(M\) in the formula:
A two-sided confidence interval splits the remaining probability equally between two tails. If the confidence level is 95%, then 5% total is outside the interval: 2.5% in each tail. The R functions use \(1 - \alpha/2\) as the quantile.
Use \(z^*\) for proportions, where the SE formula does not involve \(\sigma\). Use \(t^*\) when estimating a mean with an unknown \(\sigma\). The t-distribution has heavier tails, so \(t^*\) is always larger than \(z^*\) for the same confidence level.
As degrees of freedom increase, the t-distribution approaches the standard normal. By df = 30, the difference is small. By df = 120, they are nearly identical. The common Z critical values (1.645, 1.960, 2.576) are the limits as df approaches infinity.
Estimates a population proportion \(\pi\) using the observed sample proportion \(\hat{p}\).
Unlike hypothesis testing (which uses \(\pi_0\) in the standard error), confidence intervals have no null value. We estimate the SE using the observed \(\hat{p}\).
The normal approximation requires both \(n\hat{p} \geq 10\) and \(n(1-\hat{p}) \geq 10\).
We are \(C\%\) confident that the true population proportion lies within the interval. This means that if we repeated the sampling process many times, about \(C\%\) of the resulting intervals would contain the true parameter.
Each simulated sample draws \(k^* \sim \text{Binomial}(n, \hat{p})\) and computes a new CI. The coverage rate is the fraction of simulated intervals that contain the true value (set to the observed \(\hat{p}\)).
A \(C\%\) confidence interval contains all values of \(\pi_0\) that would not be rejected at the \(\alpha = 1 - C\) significance level.
Estimates a population mean \(\mu\) when the population standard deviation is unknown.
When \(\sigma\) is unknown and estimated by \(s\), the extra uncertainty is captured by the t-distribution. The t-distribution has heavier tails than the normal, so the interval is wider. As \(n\) grows, \(t^*\) converges to \(z^*\).
The t-interval requires approximate normality or a large enough sample. Tintle et al. use \(n \geq 20\). The traditional CLT threshold is \(n \geq 30\).
Each simulated sample draws \(n\) observations from N(\(\bar{x}\), \(s\)), computes the sample mean, sample SD, and constructs a new t-interval. The coverage rate is the fraction of intervals that contain the true mean (set to the observed \(\bar{x}\)).
A \(C\%\) confidence interval contains all values of \(\mu_0\) that would not be rejected by a two-sided t-test at the \(\alpha = 1 - C\) level.
Estimates the difference \(\pi_1 - \pi_2\) between two population proportions.
Unlike the two-proportion hypothesis test (which pools under H\(_0\): \(\pi_1 = \pi_2\)), the confidence interval uses each group's own \(\hat{p}\) in the standard error. There is no null hypothesis to assume equal proportions.
All four of the following must be \(\geq 10\): \(n_1\hat{p}_1\), \(n_1(1-\hat{p}_1)\), \(n_2\hat{p}_2\), \(n_2(1-\hat{p}_2)\).
If the interval contains 0, there is no significant difference between the two proportions at the given confidence level. This is equivalent to failing to reject H\(_0\): \(\pi_1 = \pi_2\) in a two-sided test.
Each simulation draws \(k_1^* \sim \text{Binomial}(n_1, \hat{p}_1)\) and \(k_2^* \sim \text{Binomial}(n_2, \hat{p}_2)\), then constructs a CI for the difference. The coverage rate is the fraction of intervals that contain the true difference.
Estimates the difference \(\mu_1 - \mu_2\) between two population means. Welch's version does not assume equal variances.
The pooled t-interval assumes \(\sigma_1 = \sigma_2\). Welch's interval relaxes this assumption and is the safer default. R's t.test() uses Welch by default.
Each group needs approximate normality or a large enough sample. Tintle et al. use \(n \geq 20\) per group. The traditional CLT threshold is \(n \geq 30\) per group.
Each simulation draws two independent samples from N(\(\bar{x}_1\), \(s_1\)) and N(\(\bar{x}_2\), \(s_2\)). For each pair, we compute the Welch t-interval with its own df. The coverage rate is the fraction of intervals that contain the true difference.
If the interval contains 0, there is no significant difference between the means at the given confidence level.
Estimates the mean difference \(\mu_d\) between paired observations. This reduces to a one-sample t-interval on the differences.
where \(\bar{d}\) is the mean of the differences and \(s_d\) is the standard deviation of the differences.
Pairing removes between-subject variability. Instead of comparing two independent groups, we analyze within-subject differences. This often reduces the standard error and produces a narrower interval.
Use the paired interval when observations come in natural pairs: before/after measurements on the same subject, matched subjects, or repeated measures. Use the two-sample interval when the groups are independent.
The differences need approximate normality or a large enough sample. Tintle et al. use \(n \geq 20\) pairs. The traditional CLT threshold is \(n \geq 30\).
If the interval contains 0, there is no significant mean difference at the given confidence level. This is equivalent to failing to reject H\(_0\): \(\mu_d = 0\) in a two-sided paired t-test.
Each simulated sample draws \(n\) differences from N(\(\bar{d}\), \(s_d\)) and constructs a new t-interval. The coverage rate is the fraction of intervals that contain the true mean difference.