Statistical Significance
What does "statistical significance" really mean?
In Social Behavioral Sciences it is customary to describe one's finding as statistically significant, when the obtained result is among those that (theoretically) would occur no more than 5 out of 100 (p < .05) times when the only factors operating are the chance variations that occur whenever random samples are drawn.
Procedure Used to Test for Significance
Whenever we perform a significance test, it involves comparing a test value that we have calculated to some critical value for the statistic. It does
not matter what type of statistic we are calculating (e.g., a t-statistic, a chi-square statistic, an F-statistic, etc.), the procedure to test for significance is the same.
If your statistic is higher than the critical value from the table:
- Your finding is significant.
- You reject the null hypothesis.
- The probability is small that the difference or relationship happened by chance, and p is less than the critical alpha level (p < alpha ).
If your statistic is lower than the critical value from the table:
- Your finding is not significant.
- You fail to reject the null hypothesis (or
accept the null hypothesis).
- The probability is high that the difference or relationship happened by chance, and p is greater than the critical alpha level (p > alpha ).
Today we have the advantage of using computer software
that can calculate exact probabilities for most test statistics. If you have an exact probability from computer software, simply compare it to your critical alpha level.
If the exact probability is less than the critical alpha level, your finding is significant, and if the exact probability is greater than your critical alpha level, your finding is not significant. Using a table is not necessary when you have the exact probability for a statistic. |
