The independent samples or between-subjects t-test is used to compare two (and only two) means from different groups. In an independent measures/between-subjects design, a dependent variable is measured under at least two different conditions (two levels of a single independent variable), experienced by different people in different groups. In contrast, a repeated measures/within-subjects design would involve the same people experiencing these different conditions. The independent t-test is used when the independent variable consists of two levels only, and the two levels are experienced by two different groups of different people. If there are more than two levels, the analysis of variance (ANOVA) must be used. This technique is explained in a later section.

The logic of the t-test is a bit complicated, and is explained in more detail in a companion video posted separately. Briefly, a t-test involves calculating the magnitude of the difference between two sample means, relative to the variability within each sample. An assumption is made that this difference was due to chance, and the probability of getting this difference under that assumption, given the amount of variability in the samples, is calculated. When the probability of getting these results under the assumption they were due to chance is low, the assumption is rejected. We call the assumption that the results were due entirely to chance the null hypothesis, and it’s quite different from the hypothesis of a study. In fact, most of the time the null hypothesis is the opposite of the prediction made and tested with the study. For example, we might hypothesize that a new antidepressant reduced symptoms of depression more than a placebo. If we compared average depression scores from a group of participants randomly assigned to take the antidepressant with those of other participants randomly assigned to take the placebo, we would use an independent measures/between-subjects t-test, and our null hypothesis would be that the antidepressant and placebo have the same effect, that any differences we see would be due to chance. This is counterintuitive, because we wouldn’t conduct the study if we didn’t think there would be a difference between the groups was was due to a difference in treatment and NOT to chance. Nevertheless, the statistic requires that the assumption of the null hypothesis be made, and that the t-test is conducted under this assumption. Again, the details are explained further in the video at the end of the chapter.

There are two types of independent measures/between-subjects t-tests, one- and two-tailed. Two-tailed tests are by far the more common, and are used when the experimenter does not have overwhelming evidence for specifying a direction of the effect of the independent variable. For example, the antidepressant mentioned above might actually make people more depressed (in fact, some antidepressants have been linked to suicide in certain populations). When there is good evidence, or a logical reason for reliably expecting a change in a particular direction, a one-tailed test may be used. For example, if there is no way that a particular treatment could result in the dependent measure moving in one direction, a one-tailed test might be used. One-tailed tests are rarely used correctly, however, and unless a compelling argument can be made, the two-tailed test is the default.

Hypothesis testing for independent t-tests has 4 steps:

Step 1: State the hypotheses

When setting up the two-tailed hypotheses for independent samples t-test, population means are used and are represented by µ₁andµ₂. The null hypothesis predicts there is no difference between the means of the samples, i.e. the mean difference will be equal to zero. This indicates that the samples came from the same population. The alternative hypothesis predicts there is a difference between the means of the samples, i.e. the mean difference will not be equal to zero. This indicates that the samples do not come from the same population.

If this were a one-tailed test and the researcher expected the second sample to have a larger mean than the first sample, the alternative hypothesis would have a < sign, instead of not equal. This would indicate that the mean for µ₁ would be smaller than µ₂  and when you subtract µ₂  from µ₁ you will get a negative number (i.e. less than 0). The null hypothesis would have a > sign to cover all other possible values.

H₀: µ₁ -µ₂  > 0

H₁: µ₁ -µ₂  < 0

For a one-tailed test where the researcher expected the second sample to have a smaller mean than the first sample, the the alternative hypothesis would have a > sign and the null hypothesis would have a < sign

H₀: µ₁ -µ₂  < 0

H₁: µ₁ -µ₂  > 0

Step 2: Determine the critical region

Using a t-test table (also available on pg 539 of the Morling book) and the degrees of freedom (df) for the sample, a critical value can be determined for the analysis.

For independent samples t-test, df = n₁ + n₂ – 2, where n₁ equals the number of participants in the first sample and n₂ equals the number of participants in the second sample

Step 3: Calculate your statistic

First, the pooled variance needs to be calculated. The pooled variance is an average of the variance scores for both samples. We are combining the data so that variance for each sample is contributing to the final calculated t.

Second, the estimated standard error needs to be calculated. The estimated standard error represents the average distance of each sample statistic from the population parameter. It is a measure of variability of the scores. The more scores deviate from the mean, the higher this number will be.

Finally, t can be calculated.

Essentially, you are dividing the difference between the means (i.e, the numerator) by the amount of variation in the sample (i.e. the denominator). Note that you ignore the second part of the numerator (µ₁ -µ₂) because it equals zero. It is equal to zero because it is based on the null hypothesis that the population means are equal. If you subtract one from the other, there shouldn’t be a difference. You are left with just the sample mean difference in the numerator.

The sample size for the study greatly impacts the calculated t value. The t-test equation is made up of two parts: the mean difference and a measure of variability (in this case, estimated standard error). If you have high variability in your sample, this decreases the chance you will find an effect. As variability increases, the value of t decreases. However, sample size also plays a role. As your sample size increases, your variability decreases and your chance of finding an effect increases.

Step 4: State a decision

Compare the calculated t to the critical value for t to determine whether to reject or fail to reject the null hypothesis. If the calculated t falls in your critical region (in the tails of the distribution beyond your critical value), then you reject the null hypothesis. If your calculated t does not fall within your critical region, you fail to reject the null hypothesis.

If you decide to reject the null hypothesis, an effect size should be calculated. Two different methods can be used for effect size and each should give you approximately the same answer: Cohen’s d and r².

Cohen’s d provides a standardized measure of the mean difference, similar to the way a z-score can be used as a standardized number for a sample mean.

With Cohen’s d, you use the following to interpret the calculation.

	Effect Size
0.20	Small
0.50	Medium
0.80	Large

Alternatively, you can use r², which measures the percentage of variance in the population accounted for by the sample statistics.

With r², you use the following to interpret the calculation.

r2	Effect Size
0.001	Small
0.09	Medium
0.25	Large

Next, you should write an interpretation of the results in APA style. Your interpretation should at a minimum include:

• the independent and dependent variables (i.e. what was being tested in the study)
• a narrative explanation of the statistics, and
• the statistical copy presented in APA style:  t(df) = calculated t, p < or > alpha, effect size