Using the t-test to measure student learning (paired sample)

Maria T Ping & Willy A Renandya

You have just attended a one-day workshop on how to teach speaking skills more effectively. You find the ideas interesting and are thinking of trying out the teaching techniques with your students.

The following year, you put together a set of lesson plans based on the workshop ideas and teach your students for the whole semester. On the first day of the semester, you give your students a test in order to get baseline data about their speaking ability.

As you teach your students, you notice that your students seem to be more active and motivated. They speak more, they interact more with each other and their ability to express themselves in English seem to show improvement . At the end of the semester, you give them a similar test like the one you gave earlier.

As you compare the test results, you notice that your students’ performance on the post-test is higher than on the pre-test. But how can you confidently say that the improvement is statistically significant? You need to perform a statistical test, called the t-test! We describe below the purpose and steps for running the t-statistic.

Purpose

1. The t-test is used to compare two sets of scores in order to determine whether the average score (called the mean score) of the first set is significantly higher/lower than the second set.
2. When the two sets of scores come from the same group of students (like in our example here), we use the paired-sample t-test.
3. When the two sets of scores come from two different groups of students, we use the independent-sample t-test.
4. In order to have a statistically reliable result, the number of scores in each set should be around 25- 30. It is all right to have more, but experts agree that ideally it should not be fewer than 25.  If we have a rather small sample size (e.g. n < 15), we should probably not use the t-test as the results may not be accurate.

Procedure for a paired-sample t-test

1. Prepare your sets of scores (two sets of mean scores, no more and no less)
2. If you are a Math Geek, you might want to take your own sweet time and calculate manually by entering the values into the paired-sample t-test formula. However, if you want to get the result rather quickly- yet still very much accurately- we recommend using free online statistical tools such as VassarStats and Social Science Statistics (links are provided below).
3. Interpret your results and conclude whether there is a significant difference between the two sets of scores or not. The rules of thumb: If your t computed is larger than the critical t/ t-table and your probability (p) value is less than 0.05, you can conclude that the difference is statistically significant.
4. There are a few terms you need to know with when dealing with paired-sample t-test:
   a. t computed: the t-value that you get from your calculation
   b. critical t value à the value that we use to compare to our t computed, in order to find out if there is a statistically significant difference or not
   c. degree of freedom (df) à for paired sample t-test, df = sample size (N) – 1. e.g. your sample size is 30, then your df= 29
   d. Level of significance à the most commonly used one is 0.05 (5%)
   e. Probability (p) value à this value must be less than your level of significance in order to conclude that there is a significant difference

Click here to see an example of how the paired sample t-test is done using Socsstatistics

Useful links:
https://www.socscistatistics.com/tests/ttestdependent/default.aspx
http://vassarstats.net/tu.html
https://www.danielsoper.com/statcalc/default.aspx
https://statistics.laerd.com/spss-tutorials/dependent-t-test-using-spss-statistics.php
http://www.statstutor.ac.uk/resources/uploaded/paired-t-test.pdf