Chapter Notes: Comparing Two Means

Table of Contents
1. Understanding the Two-Sample Problem
2. Conditions for Comparing Two Means
3. The Standard Error of the Difference
4. Hypothesis Testing for Two Means
5. Confidence Intervals for the Difference Between Two Means
6. Relationship Between Hypothesis Tests and Confidence Intervals
7. Pooled vs. Unpooled Procedures
8. Practical Considerations and Common Mistakes
9. Summary
View more

When we want to understand whether two groups are truly different from each other, we often compare their average values. For instance, do students who use a new study method score higher than those who use traditional methods? Does a new medication reduce blood pressure more than a placebo? These questions require us to compare the means (averages) of two separate groups. However, because we work with samples rather than entire populations, we must account for variability and determine whether observed differences are likely due to real effects or just random chance. This process involves hypothesis testing, confidence intervals, and careful consideration of the conditions under which our methods are valid.

Understanding the Two-Sample Problem

When we compare two means, we are working with data from two independent samples. Each sample comes from its own population, and we want to know if the population means differ. For example, we might measure the heights of adult males and adult females, the fuel efficiency of two car models, or the test scores of students from two different teaching methods.

The key features of a two-sample means problem include:

• Independence between groups: The two samples are collected separately, and measurements in one group do not affect measurements in the other group.
• Two population means: We denote the mean of population 1 as \( \mu_1 \) and the mean of population 2 as \( \mu_2 \).
• Sample statistics: From sample 1, we calculate \( \bar{x}_1 \), \( s_1 \), and \( n_1 \) (mean, standard deviation, and sample size). From sample 2, we have \( \bar{x}_2 \), \( s_2 \), and \( n_2 \).
• The parameter of interest: We focus on \( \mu_1 - \mu_2 \), the difference between the two population means.

The estimator for \( \mu_1 - \mu_2 \) is \( \bar{x}_1 - \bar{x}_2 \), the difference between the two sample means. This statistic has its own sampling distribution with its own mean and standard deviation (called the standard error).

Conditions for Comparing Two Means

Before performing inference on two means, we must verify that certain conditions are met. These conditions ensure that our methods produce reliable results.

Independence Conditions

We need two types of independence:

• Independence within each sample: Observations within each group must be independent of one another. This is typically satisfied if data are collected using random sampling or random assignment in an experiment.
• Independence between samples: The two samples must be independent of each other. This means the data collection process for one group does not influence the other. For example, measuring the reaction time of one group of participants should not affect the reaction times measured in a different group.

If the same individuals are measured twice (like before-and-after measurements), the samples are paired or dependent, and we must use different methods (paired t-test) instead of the two-sample procedures discussed here.

Normality Condition

The sampling distribution of \( \bar{x}_1 - \bar{x}_2 \) should be approximately normal. This happens when:

• Both populations are normally distributed, OR
• Both sample sizes are large enough (typically \( n_1 \geq 30 \) and \( n_2 \geq 30 \)) so that the Central Limit Theorem applies, OR
• For smaller samples, the data in each group show no strong skewness or outliers.

We can check this condition by examining histograms, boxplots, or normal probability plots of each sample.

Sample Size Condition

For the methods to work reliably:

• Each sample should be less than 10% of its respective population (the 10% condition). This ensures independence when sampling without replacement.
• Larger samples generally produce more reliable results and allow us to detect smaller differences.

The Standard Error of the Difference

The standard error measures how much we expect \( \bar{x}_1 - \bar{x}_2 \) to vary from sample to sample. When the two samples are independent, the variance of their difference is the sum of their individual variances. Therefore, the standard error is:

Here, \( s_1 \) and \( s_2 \) are the sample standard deviations, and \( n_1 \) and \( n_2 \) are the sample sizes. This formula assumes the two population variances may be different, which is the most common and safest assumption.

Think of this like combining uncertainty from two sources: just as errors in two separate measurements add up when we subtract the measurements, the variability in two sample means combines when we look at their difference.

Hypothesis Testing for Two Means

A hypothesis test helps us determine whether the observed difference between sample means is statistically significant-that is, unlikely to occur by chance alone if the population means were actually equal.

Setting Up the Hypotheses

The null hypothesis typically states that there is no difference between the population means:

The alternative hypothesis expresses what we are trying to find evidence for. It can take three forms:

• Two-sided: \( H_a: \mu_1 - \mu_2 \neq 0 \) (the means are different, but we don't specify which is larger)
• One-sided (right-tailed): \( H_a: \mu_1 - \mu_2 > 0 \) (population 1 has a larger mean than population 2)
• One-sided (left-tailed): \( H_a: \mu_1 - \mu_2 < 0="" \)="" (population="" 1="" has="" a="" smaller="" mean="" than="" population="">

The Two-Sample t-Test

The test statistic for comparing two means is:

The zero in the numerator represents the hypothesized difference under \( H_0 \). This t-statistic measures how many standard errors the observed difference is from zero.

The degrees of freedom for this test are calculated using a complex formula (the Welch-Satterthwaite approximation):

Most statistical software and calculators compute this automatically. The result is usually not a whole number and is rounded down to the nearest integer.

Finding the P-Value

The p-value is the probability of observing a test statistic as extreme as (or more extreme than) the one calculated, assuming the null hypothesis is true. We find it using the t-distribution with the calculated degrees of freedom:

• For a two-sided test: p-value = \( 2 \times P(T \geq |t|) \)
• For a right-tailed test: p-value = \( P(T \geq t) \)
• For a left-tailed test: p-value = \( P(T \leq t) \)

Making a Decision

We compare the p-value to the significance level \( \alpha \) (commonly 0.05):

• If p-value < \(="" \alpha="" \),="" we="">reject the null hypothesis. We have sufficient evidence that the population means differ.
• If p-value ≥ \( \alpha \), we fail to reject the null hypothesis. We do not have sufficient evidence that the population means differ.

Example:  A researcher wants to determine if a new teaching method improves test scores.
She randomly assigns 25 students to the new method (Group 1) and 28 students to the traditional method (Group 2).
The new method group has a mean score of 78.4 with a standard deviation of 8.2.
The traditional method group has a mean score of 74.1 with a standard deviation of 9.5.

Test at the 0.05 significance level whether the new method produces higher scores.

Solution:

Step 1: State the hypotheses.
\( H_0: \mu_1 - \mu_2 = 0 \) (no difference in mean scores)
\( H_a: \mu_1 - \mu_2 > 0 \) (new method has higher mean scores)
This is a right-tailed test.

Step 2: Check conditions.
Independence: Students were randomly assigned to groups (satisfied).
Normality: Sample sizes are reasonably large (\( n_1 = 25 \), \( n_2 = 28 \)), so the Central Limit Theorem applies.
Sample size: Random assignment ensures independence (satisfied).

Step 3: Calculate the standard error.
\( SE = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}} = \sqrt{\frac{8.2^2}{25} + \frac{9.5^2}{28}} \)
\( SE = \sqrt{\frac{67.24}{25} + \frac{90.25}{28}} = \sqrt{2.6896 + 3.2232} = \sqrt{5.9128} \approx 2.43 \)

Step 4: Calculate the test statistic.
\( t = \frac{\bar{x}_1 - \bar{x}_2}{SE} = \frac{78.4 - 74.1}{2.43} = \frac{4.3}{2.43} \approx 1.77 \)

Step 5: Find degrees of freedom (using calculator or software).
Using the Welch-Satterthwaite formula: \( df \approx 50.7 \), which we round down to 50.

Step 6: Find the p-value.
Using a t-table or technology with \( df = 50 \) and \( t = 1.77 \) for a right-tailed test: p-value \( \approx 0.042 \)

Step 7: Make a decision.
Since p-value (0.042) < \(="" \alpha="" \)="" (0.05),="" we="" reject="" the="" null="">

Step 8: State the conclusion.
There is sufficient evidence at the 0.05 significance level to conclude that the new teaching method produces higher mean test scores than the traditional method.

Confidence Intervals for the Difference Between Two Means

A confidence interval provides a range of plausible values for \( \mu_1 - \mu_2 \). Unlike a hypothesis test, which gives a yes-or-no answer about a specific hypothesized value, a confidence interval estimates the actual size of the difference.

The Formula

The confidence interval for \( \mu_1 - \mu_2 \) is:

Where:

• \( \bar{x}_1 - \bar{x}_2 \) is the observed difference in sample means
• \( t^* \) is the critical value from the t-distribution with the appropriate degrees of freedom, corresponding to the desired confidence level
• \( SE = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}} \) is the standard error

Interpreting the Confidence Interval

If we construct a 95% confidence interval, we can say: "We are 95% confident that the true difference between population means (\( \mu_1 - \mu_2 \)) falls within this interval." This means that if we repeated the sampling process many times and constructed a confidence interval each time, about 95% of those intervals would contain the true difference.

Key observations:

• If the interval contains zero, it suggests that the two population means might be equal (we would not reject \( H_0 \) at that confidence level).
• If the entire interval is positive, it suggests \( \mu_1 > \mu_2 \).
• If the entire interval is negative, it suggests \( \mu_1 < \mu_2="">

Example:  A nutritionist compares the average daily calorie intake of vegetarians and non-vegetarians.
A random sample of 35 vegetarians has a mean intake of 1850 calories with a standard deviation of 240 calories.
A random sample of 40 non-vegetarians has a mean intake of 2100 calories with a standard deviation of 310 calories.

Construct a 95% confidence interval for the difference in mean calorie intake (vegetarians - non-vegetarians).

Solution:

Step 1: Identify the given information.
Group 1 (vegetarians): \( \bar{x}_1 = 1850 \), \( s_1 = 240 \), \( n_1 = 35 \)
Group 2 (non-vegetarians): \( \bar{x}_2 = 2100 \), \( s_2 = 310 \), \( n_2 = 40 \)
Confidence level: 95%

Step 2: Calculate the difference in sample means.
\( \bar{x}_1 - \bar{x}_2 = 1850 - 2100 = -250 \) calories

Step 3: Calculate the standard error.
\( SE = \sqrt{\frac{240^2}{35} + \frac{310^2}{40}} = \sqrt{\frac{57600}{35} + \frac{96100}{40}} \)
\( SE = \sqrt{1645.71 + 2402.50} = \sqrt{4048.21} \approx 63.6 \) calories

Step 4: Find degrees of freedom and critical value.
Using the Welch-Satterthwaite formula (via calculator): \( df \approx 71 \)
For 95% confidence and \( df = 71 \), \( t^* \approx 1.994 \)

Step 5: Calculate the margin of error.
Margin of error = \( t^* \times SE = 1.994 \times 63.6 \approx 126.9 \) calories

Step 6: Construct the confidence interval.
\( (\bar{x}_1 - \bar{x}_2) \pm \text{margin of error} = -250 \pm 126.9 \)
Lower bound: \( -250 - 126.9 = -376.9 \) calories
Upper bound: \( -250 + 126.9 = -123.1 \) calories
Confidence interval: \( (-376.9, -123.1) \) calories

Interpretation: We are 95% confident that the true mean daily calorie intake for vegetarians is between 123.1 and 376.9 calories lower than that for non-vegetarians.

Relationship Between Hypothesis Tests and Confidence Intervals

Hypothesis tests and confidence intervals are closely related. For a two-sided test at significance level \( \alpha \), if a \( (1-\alpha) \times 100\% \) confidence interval for \( \mu_1 - \mu_2 \) does not contain zero, we would reject \( H_0: \mu_1 = \mu_2 \) at that significance level.

For example, if a 95% confidence interval for \( \mu_1 - \mu_2 \) is (2.3, 8.7), we can conclude that at the 0.05 significance level, there is a significant difference between the means because zero is not in the interval.

Confidence intervals provide more information than hypothesis tests because they give a range of plausible values for the parameter, not just a decision about whether to reject a specific null hypothesis.

Pooled vs. Unpooled Procedures

The methods described so far use the unpooled (or separate-variance) approach, which does not assume the two populations have equal variances. This is also called Welch's t-test.

An alternative is the pooled two-sample t-test, which assumes \( \sigma_1^2 = \sigma_2^2 \) (equal population variances). In this case, we combine (pool) the two sample variances into a single estimate:

The standard error becomes:

The degrees of freedom for the pooled procedure are simpler: \( df = n_1 + n_2 - 2 \).

However, the pooled procedure is less robust. If the assumption of equal variances is violated, the results can be misleading. The unpooled (Welch's) procedure is generally recommended because it does not require the equal variance assumption and performs well even when the variances are equal.

Practical Considerations and Common Mistakes

Sample Size and Power

Larger samples provide more precise estimates and greater power-the ability to detect a real difference when one exists. Small samples may fail to detect meaningful differences simply because the variability is too large.

Statistical vs. Practical Significance

A statistically significant result does not automatically mean the difference is large or important. With very large samples, even tiny differences can be statistically significant. Always consider the effect size-the actual magnitude of the difference-in addition to the p-value.

Common Errors

• Using paired methods for independent samples (or vice versa): If the same subjects are measured twice, use paired t-tests. If two separate groups are measured, use two-sample tests.
• Ignoring conditions: Always check independence, normality (or sample size), and the 10% rule before proceeding with inference.
• Misinterpreting p-values: The p-value is not the probability that \( H_0 \) is true. It is the probability of observing data as extreme as yours, assuming \( H_0 \) is true.
• Confusing confidence level with confidence interval: The confidence level (like 95%) describes the long-run success rate of the method. A specific interval either contains the parameter or it doesn't-we just don't know which.

Summary

Comparing two means is a fundamental technique in statistics for determining whether two groups differ in a meaningful way. The key steps are:

1. Verify that conditions (independence, normality or large sample size) are met.
2. Calculate the standard error of the difference using \( SE = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}} \).
3. For hypothesis testing, compute the t-statistic, find the p-value, and compare it to the significance level.
4. For estimation, construct a confidence interval using \( (\bar{x}_1 - \bar{x}_2) \pm t^* \times SE \).
5. Interpret results carefully, considering both statistical and practical significance.

These methods provide powerful tools for making informed decisions based on data from two independent samples, whether in scientific research, business analytics, public health, or social sciences.