Welch’s vs Student’s t-Test: When Variances Differ

The independent-samples t-test is one of the most-used tools in empirical research: you have two groups and want to know whether their means differ by more than sampling noise would predict. But there are two versions of this test, and the one most textbooks teach by default — Student's t-test — rests on an assumption that real data routinely violate. This article explains the difference between Student's and Welch's t-test, why modern methodologists recommend Welch's as the default, and why choosing between them with a preliminary variance test is a statistical mistake.

The two tests and the assumption that separates them

Both tests compare two group means with the same basic logic: a difference in means divided by an estimate of its standard error. They differ entirely in how that standard error — and the reference distribution — is constructed.

Student's t-test assumes homogeneity of variance: it treats the two population variances as equal and therefore pools the sample variances into a single estimate. With sample sizes n₁ and n₂, the pooled variance is a weighted average of the two, and the test has exactly n₁ + n₂ − 2 degrees of freedom.

Welch's t-test drops the equal-variance assumption. It estimates each group's standard error separately and never pools. Because the resulting statistic no longer follows an exact t-distribution with a clean integer df, Welch approximates it with the Welch–Satterthwaite equation, which typically returns a non-integer df (you will often see df reported as something like 37.4). That fractional df is the visible signature of a Welch test.

Why the pooled assumption is fragile

When the population variances really are equal, pooling is efficient and both tests give nearly identical answers. The problem is the asymmetric case. When the larger sample is paired with the smaller variance, Student's test becomes liberal — its true Type I error rate climbs well above the nominal 5%, so you reject true nulls too often. When the larger sample is paired with the larger variance, the test turns conservative and loses power. Equal sample sizes soften but do not eliminate the distortion, and perfectly balanced designs are rare.

The modern recommendation: use Welch's by default

The influential paper by Delacre, Lakens, and Leys (2017) makes the case bluntly: researchers should use Welch's t-test by default for comparing two independent means. Their reasoning has two parts.

• Welch's controls the Type I error rate across a wide range of conditions — even when both the variances and the sample sizes are unequal, the scenario where Student's test fails most badly.
• Welch's costs almost nothing when variances are equal. In the one situation Student's test was designed for, Welch's gives virtually the same p-value with only a negligible loss of power. There is no realistic downside to defaulting to it.

In short, Welch's is safer in the bad case and essentially free in the good case. That asymmetry of consequences makes it the rational default rather than a fallback you reach for only after spotting a problem.

Why you should NOT pre-test variances with Levene's

A very common workflow is: run Levene's test (or an F-test) for equal variances first, then use Student's if it is non-significant and Welch's if it is significant. This two-stage conditional procedure seems prudent but is harmful. You let a random preliminary result decide which test to run, and that conditioning distorts the overall error rate you actually report. Levene's test also has low power in small samples — exactly when unequal variances matter most — so it often fails to flag a real problem and routes you into the wrong test. The cleaner solution is to skip the gatekeeper and run Welch's unconditionally. The same logic — don't condition your main analysis on a noisy assumption check — applies when checking normality the right way.

A worked example

Suppose a researcher compares a reading-intervention group with a control on a comprehension score.

• Group A (intervention): n₁ = 30, mean = 78.0, SD = 8.0
• Group B (control): n₂ = 22, mean = 72.5, SD = 13.0

The variances are clearly unequal (64 vs 169) and the sample sizes differ, so this is exactly where Student's test misbehaves. The Welch statistic is the mean difference (5.5) divided by the unpooled standard error √(8²/30 + 13²/22) ≈ 3.13, giving t ≈ 1.76. Welch–Satterthwaite returns roughly df ≈ 32.5 — non-integer, as expected — and p ≈ 0.089, two-tailed.

For the effect size, Cohen's d standardizes the raw difference. Using a pooled SD of about 10.4, d ≈ 0.53, a medium effect, with an approximate 95% CI of [−0.03, 1.08]. The CI for d crosses zero, consistent with the non-significant p-value, yet the point estimate still suggests a practically meaningful difference worth replicating with a larger sample.

How to report it

Report all four ingredients: the statistic, the Welch-adjusted (non-integer) df, the p-value, and the effect size with its confidence interval. For example: "Welch's t(32.5) = 1.76, p = .089, d = 0.53, 95% CI [−0.03, 1.08]." Always pair the test with an effect size — significance alone says little about magnitude. Our guide on reporting effect size in APA style covers the conventions.

Comparison table

Key takeaways

• Student's t-test assumes equal population variances and pools them; Welch's does not and adjusts df via the Welch–Satterthwaite equation, giving a non-integer df.
• Make Welch's your default: it controls Type I error when variances and sample sizes are unequal, and loses almost nothing when they are equal (Delacre, Lakens & Leys, 2017).
• Do not choose the test from a preliminary Levene's or F-test — the conditional two-stage procedure distorts your error rate.
• Report t, the adjusted df, p, and Cohen's d with its CI.
• With more than two groups, the same variance logic extends to the choice between ANOVA and Kruskal–Wallis.

In MindStat, the independent-samples t-test applies this guidance automatically — defaulting to Welch's unless an equal-variance check clearly supports pooling — and reports the adjusted df, p-value, and Cohen's d with its CI.