T-Test Calculator: One-Sample, Two-Sample & Paired (Welch)

Ten runners each race a 5K twice, once in their old shoes and once in a new pair. Analysed as two separate groups of times, the new shoes look like nothing: p = 0.595. Analysed as pairs, which is what they are, the same twenty numbers give p = 0.000027. Nothing about the data changed. Only the choice of t-test did, and that choice is where t-tests go wrong.

This calculator runs all three t-tests: one sample against a fixed value, two independent samples (Welch’s or pooled) and paired measurements. Paste raw data or type the mean, standard deviation and sample size. It gives t, the degrees of freedom, the p-value for a two-sided or one-sided alternative, a confidence interval for the difference, Cohen’s d, a warning when a value looks like an outlier, and every step with your numbers. It is set to the runners’ times.

T-Test Calculator

μ₀ is the value you are testing against: a label claim, a target or a past average.

t-test –
Your samples
Where your t falls (dashed = critical value)
Step by step

    Same People, Different People or a Fixed Number?

    Three questions pick the test. Answer them before you look at any p-value.

    What you are comparingExampleTestDegrees of freedom
    One sample against a set value12 cereal boxes against the 500 g on the labelOne-sample t-testn − 1
    Two groups of different people or thingsReaction times on two kinds of monitorTwo-sample t-test (Welch’s by default)Welch formula, or n₁ + n₂ − 2 if pooled
    The same people or things measured twiceEach runner in old and new shoesPaired t-testpairs − 1

    The paired test is a one-sample test in disguise. It subtracts each pair, then asks whether the average difference is zero. That is why it needs one second value for every first value, in the same order, and why the calculator refuses two boxes of different lengths on the Paired tab.

    Two more cases need a different tool. With more than two groups, use ANOVA rather than a string of t-tests. When you know the population standard deviation, not just the sample’s, the z-test applies; in practice that almost never happens, which is why the t-test exists.

    Ten Runners, Two Shoes: Why Pairing Changes Everything

    Here are the runners’ 5K times in minutes:

    Runner12345678910
    Old shoes24.827.122.630.426.028.723.925.529.226.8
    New shoes24.126.522.329.625.627.823.624.728.926.1
    Old − new0.70.60.30.80.40.90.30.80.30.7

    Every runner was faster in the new shoes, by between 0.3 and 0.9 minutes. The paired test works only on that last row:

    1. Mean difference d̄ = 0.58 minutes, about 35 seconds.
    2. SD of the differences s = 0.2348.
    3. Standard error = 0.2348 ÷ √10 = 0.0742.
    4. t = 0.58 ÷ 0.0742 = 7.81, with 10 − 1 = 9 degrees of freedom.
    5. p-value (two-sided) = 0.000027. The 95% confidence interval for the improvement is 0.41 to 0.75 minutes, or roughly 25 to 45 seconds.

    Now treat the same numbers as two unrelated groups. The old-shoe times average 26.50 with SD 2.45; the new-shoe times average 25.92 with SD 2.35. The same 0.58-minute gap now sits against a spread of about 2.4 minutes, because runner 4 and runner 3 differ by nearly 8 minutes in either pair of shoes. Welch’s test gives t = 0.54 and p = 0.595.

    The difference between runners is real, but it has nothing to do with the shoes. Pairing removes it: each runner is compared only with themselves. The correlation between the two columns here is 0.996, and the higher that correlation, the more a paired design gains. The calculator shows what ignoring the pairing would give, next to the paired result, so you can see how much the design is doing.

    The reverse mistake also happens. Running a paired test on two groups that aren’t matched invents a pairing that doesn’t exist, and the result depends on the arbitrary order you typed them in.

    Two Separate Groups: Monitor Refresh Rate and Reaction Time

    Suppose 9 players do a reaction test on a 60 Hz monitor and 12 different players do it on a 144 Hz monitor. Times in milliseconds:

    • 60 Hz: 262, 248, 281, 255, 239, 296, 270, 251, 274 (mean 264.0, SD 17.97)
    • 144 Hz: 236, 229, 251, 244, 233, 240, 226, 247, 238, 231, 249, 242 (mean 238.83, SD 8.10)

    Welch’s test, step by step:

    1. Difference in means: 264.0 − 238.83 = 25.17 ms.
    2. Each group’s share of the variance: s₁² ÷ n₁ = 323 ÷ 9 = 35.89, and s₂² ÷ n₂ = 65.61 ÷ 12 = 5.47.
    3. Standard error: √(35.89 + 5.47) = 6.43.
    4. t = 25.17 ÷ 6.43 = 3.91.
    5. Degrees of freedom from the Welch–Satterthwaite formula: (35.89 + 5.47)² ÷ (35.89² ÷ 8 + 5.47² ÷ 11) = 10.45. It doesn’t have to be a whole number.
    6. p-value (two-sided) = 0.0027. The 95% interval for the difference is 10.9 to 39.4 ms.

    Notice how far the degrees of freedom fell. There are 21 players, but the 60 Hz group’s larger spread and smaller size dominate the standard error, so the test behaves almost as if it had only that group’s 9 values. A whole-number shortcut of n₁ + n₂ − 2 = 19 would overstate the evidence.

    To run this yourself, open the Two samples tab and paste each list into its box. One player per value, no player in both groups.

    Welch or Pooled? What the Choice Does to Your Error Rate

    The pooled (Student’s) test assumes both groups come from populations with the same standard deviation and averages the two sample variances into one. Welch’s test drops that assumption. For the monitor data, pooling gives t = 4.33 with 19 degrees of freedom and p = 0.00036, seven times smaller than Welch’s 0.0027. Here one SD is 2.2 times the other, so the pooled answer is too confident.

    How much does it matter in general? A simulation of 100,000 experiments with no real difference between the groups, testing at α = 0.05, shows it:

    Group sizes and SDsPooled test: false positivesWelch’s test: false positives
    Equal: 20 and 20, SD 10 and 105.0%5.0%
    Small group has the big SD: 8 with SD 20, 40 with SD 533.9%5.0%
    Big group has the big SD: 40 with SD 20, 8 with SD 50.03%5.1%

    When the groups are similar, both tests agree. When the smaller group is the more variable one, the pooled test calls a difference “significant” about one time in three when there is none. In the reverse case it almost never rejects, so it misses real effects. Welch’s test stays near the 5% it promises in all three. That is why the calculator uses Welch’s test unless you choose otherwise, and why it shows what the other method would have given.

    One habit is to run a test for equal variances first and pick the t-test from that result. The variance test has little power in small samples, exactly where the choice matters, so defaulting to Welch’s test is simpler and safer.

    Testing Against a Fixed Number: Are the Boxes Underfilled?

    A cereal box says 500 g. You weigh 12 boxes: 497, 502, 495, 499, 493, 498, 501, 494, 496, 500, 492 and 497 grams.

    1. Mean x̄ = 497.0 g, SD s = 3.162 g, n = 12.
    2. Standard error = 3.162 ÷ √12 = 0.913.
    3. t = (497.0 − 500) ÷ 0.913 = −3.29, with 11 degrees of freedom.
    4. p-value, two-sided: 0.0073. The 95% interval for the true mean is 494.99 to 499.01 g, which stops short of 500.

    If the question had been set up beforehand as “are the boxes underfilled?”, the one-sided test (alternative: less) gives p = 0.0036, half the two-sided value. The one-sided p is smaller only because it ignores the possibility of overfilling. That’s fine when overfilling genuinely doesn’t matter to the question and the direction was chosen before weighing. Choosing it after seeing the mean is below 500 doubles the real false-positive rate.

    Typing Numbers From a Paper: SD or Standard Error?

    The Summary option lets you run a t-test from a report that gives only means, SDs and sample sizes. Check which spread the report gives. Many tables print “mean ± SE” rather than “mean ± SD”, and the two differ by a factor of √n.

    The monitor groups have SDs of 17.97 and 8.10, but standard errors of 5.99 and 2.34. Type the standard errors into the SD boxes and t jumps from 3.91 to 11.94, with p falling from 0.0027 to about 0.0000004. The difference looks overwhelming only because the spread was shrunk twice. If a “SD” looks small next to the range of the data, or the caption says SEM, multiply it by √n before typing it in.

    For paired data, the summary you need is the SD of the differences. The two separate SDs of “before” and “after” can’t give it, because they say nothing about how closely the pairs move together.

    One Bad Value Can Make the Result Weaker, Not Stronger

    Suppose box 4 was crushed and weighs 470 g instead of 499. The mean drops to 494.58 g, further from 500 than before, so you might expect stronger evidence of underfilling. The opposite happens: the SD jumps from 3.16 to 8.34, t shrinks to −2.25 and p rises from 0.0073 to 0.046.

    The t statistic divides the gap by the spread, and a single extreme value inflates the spread more than it moves the mean. In the other direction, an outlier on the far side can create a difference that isn’t there. Neither is a reason to delete values you don’t like. The right move is to check whether the value is a typing slip, a measurement error or a genuine observation, and report what you did.

    The calculator marks any value outside the 1.5 × IQR fences (more than one and a half times the interquartile range beyond the quartiles) and says so in the tip. With raw data it also flags strong skew in small samples, where the t-test’s p-value is only approximate.

    How Big Is the Difference? Three Versions of Cohen’s d

    A p-value says whether a difference is distinguishable from zero. Cohen’s d says how big it is, in standard-deviation units, so results on different scales can be compared. The catch is that there’s more than one d, and the right one depends on the design:

    • Two independent groups: d = difference ÷ pooled SD. For the monitors, 25.17 ÷ 13.19 = 1.91.
    • One sample: d = (x̄ − μ₀) ÷ s. For the cereal, (497 − 500) ÷ 3.162 = −0.95.
    • Paired data: d<sub>z</sub> = mean difference ÷ SD of the differences. For the runners, 0.58 ÷ 0.2348 = 2.47.

    The runners show why the label matters. Divide the same 0.58 minutes by the runners’ own spread of times (about 2.4 minutes) instead, and you get 0.24. Both numbers are correct; they answer different questions. d<sub>z</sub> = 2.47 describes how consistently the shoes helped each runner. The 0.24 describes how big 35 seconds is next to the normal range of 5K times. A report should say which one it gives.

    Jacob Cohen’s rough labels are 0.2 small, 0.5 medium and 0.8 large, offered as a starting point for fields with no better benchmark. With small samples d runs slightly high, so the calculator also gives Hedges’ g, which shrinks it a little (for the monitors, 1.83 instead of 1.91).

    Reading a Non-Significant Result

    A p-value above 0.05 is not evidence that two means are equal. It means the data couldn’t rule out a difference of zero. The confidence interval shows which differences remain possible.

    Take the paired example from Microsoft’s documentation of Excel’s T.TEST function: two columns of nine values each, giving a two-sided p of 0.196. The mean difference is −3.33 with a 95% interval of about −8.8 to +2.1. A difference of zero is inside that range, but so is a difference of −8, which in many settings would matter a great deal. “No significant difference” here really means “not enough data to tell”.

    To show that two means are close enough to treat as the same, the question has to be set up the other way round: decide on the largest difference that wouldn’t matter, then test whether the whole confidence interval fits inside it. This is called an equivalence test (two one-sided tests, or TOST). A t-test that simply fails to reach significance can’t do that job.

    Before and After Isn’t Proof of Cause

    A paired test on before-and-after measurements shows that the numbers changed. It doesn’t show why. People measured twice can improve because they practised the test, because the weather changed, because the worst scores at the start drift back toward average (regression to the mean), or because they knew what was being tested.

    The runners are a fair example: the second race might have been on a cooler day. Running half the group in the new shoes first and half in the old shoes first spreads that kind of effect across both. For a treatment, the stronger design is a control group that gets measured at the same times without the treatment. Comparing the two groups’ changes then uses a two-sample test on the differences.

    t, Degrees of Freedom and the Critical Value

    The t statistic is always the same shape: an observed difference divided by its standard error. What changes from test to test is how the standard error and the degrees of freedom are worked out.

    The degrees of freedom set the exact t-distribution the statistic is compared with. Small samples get heavier tails, so the same t produces a larger p-value. At 9 degrees of freedom, a two-sided test at α = 0.05 needs |t| ≥ 2.262. At 30 degrees of freedom it needs 2.042, and only with unlimited data does it come down to the normal distribution’s 1.960. The t table lists these critical values, and the calculator shows the one for your test.

    Comparing t with the critical value and comparing p with α always give the same decision. The critical value is handy for checking by hand; the p-value carries more information, because it says how far past the line you are.

    Assumptions, and How Much They Matter

    All three t-tests share a short list of assumptions:

    • Independent observations. Each value comes from a different person or item, apart from the deliberate pairing in a paired test. Ten readings from the same person are not ten observations. When readings from the same source are alike, treating them as separate makes p too small.
    • Roughly normal populations. For a paired test, it’s the differences that need to be roughly normal, not the raw scores. The t-test tolerates moderate departures, more so as the sample grows, because the sample mean becomes close to normal even when the data aren’t. With small samples and strong skew or outliers, a rank-based test such as the Wilcoxon or Mann–Whitney test is a sensible check.
    • Equal standard deviations for the pooled test only. Welch’s test doesn’t need it.

    The measurement scale matters too. A t-test compares means, so it needs data where an average makes sense. For counts in categories, use a chi-square test instead.

    Checking a t-Test in Excel or Google Sheets

    Spreadsheets give the p-value directly with T.TEST:

    What you wantFormula
    Paired test, two-sided=T.TEST(A2:A11, B2:B11, 2, 1)
    Two samples, equal variances (pooled)=T.TEST(A2:A10, B2:B13, 2, 2)
    Two samples, unequal variances (Welch’s)=T.TEST(A2:A10, B2:B13, 2, 3)
    One-sided version of any of thesechange the third argument to 1
    Critical value, two-sided=T.INV.2T(0.05, df)

    There is no one-sample option. To test the cereal boxes against 500, put the weights in one column and 500 in every row of another, then run the paired test (type 1).

    Two details trip people up. First, T.TEST returns only the p-value, not t, the degrees of freedom or a confidence interval. Second, the one-tailed version works with the size of t but ignores its sign, so it returns a p-value below 0.5 even when the data point the opposite way from your hypothesis. Check the direction of the means yourself. For the runners, =T.TEST(old, new, 2, 1) returns about 0.000027, matching the calculator.

    Writing Up the Result

    A complete report names the test, gives t with its degrees of freedom, the p-value, the effect size and ideally a confidence interval. In APA style:

    • Runners: a paired t-test showed faster times in the new shoes, t(9) = 7.81, p < .001, d<sub>z</sub> = 2.47, mean improvement 0.58 min, 95% CI [0.41, 0.75].
    • Monitors: Welch’s t-test, t(10.45) = 3.91, p = .003, d = 1.91, difference 25.2 ms, 95% CI [10.9, 39.4].
    • Cereal: one-sample t-test against 500 g, t(11) = −3.29, p = .007, d = −0.95.

    Keep the fractional degrees of freedom for Welch’s test rather than rounding them to a whole number, and say whether the test was one-sided. The calculator’s Copy button puts t, df, p, d and the interval on one line, and the p-value calculator turns any t and df from a paper back into a p-value.

    Before You Collect Data: How Many in Each Group?

    A t-test can only find a difference the sample is large enough to see. The usual planning target is 80% power: an 80% chance of a significant result if the true effect is the size you expect. At α = 0.05, two-sided:

    Expected effect size dPer group, two independent groups (80% power)Per group, 90% powerPairs, paired test (80% power)
    0.2 (small)394527199
    0.5 (medium)648634
    0.8 (large)263415
    1.0172310

    Two patterns stand out. Halving the effect size roughly quadruples the sample, and a paired design needs about half as many pairs as a two-group design needs people in each group for the same d (for the paired column, d is d<sub>z</sub>, so the gain is larger still when the pairs are strongly correlated). If you plan to estimate a mean rather than test a difference, the sample size calculator works out the number from the margin of error you need instead.

    A study that finds nothing with 10 per group, when the plausible effect is medium, hasn’t shown there is no effect. It had about an 18% chance of finding one.