Introduction to Power

Prerequisites

• Significance Testing, Type I and II Errors, Misconceptions

Define power

• Suppose you work for a foundation whose mission is to support researchers in mathematics education and your role is to evaluate grant proposals and decide which ones to fund.
• You receive a proposal to evaluate a new method of teaching high-school algebra.
• The research plan is to compare the achievement of students taught by the new method with those taught by the traditional method.
• The proposal contains good theoretical arguments why the new method should be superior and the proposed methodology is sound.
• In addition to these positive elements, there is one important question still to be answered: Does the experiment have a high probability of providing strong evidence that the new method is better than the standard method even if, in fact, the new method is actually better?
• It is possible, for example, that the proposed sample size is so small that even a fairly large population difference would be difficult to detect.
• That is, if the sample size is small, then even a fairly large difference in sample means might not be significant.
• If the difference is not significant, then no strong conclusions can be drawn about the population means.
• It is not justified to conclude that the null hypothesis that the population means are equal is true just because the difference is not significant.
• Of course, it is not justified to conclude that this null hypothesis is false. Therefore, when an effect is not significant, the result is inconclusive.
• You may prefer that your foundation's money be used to fund a project that has a higher probability of being able to make a strong conclusion.

• Power is defined as the probability of correctly rejecting a false null hypothesis.
• In terms of our example, it is the probability that given there is a difference between the population means of the new method and the standard method, the sample means will be significantly different.
• The probability of failing to reject a false null hypothesis is often referred to as β.
• Therefore power can be defined as:

:${\mbox{power}}=\mathbb {P} {\big (}{\mbox{reject null hypothesis}}{\big |}{\mbox{null hypothesis is false}}{\big )}$

               power = 1 - β.

Identify situations in which it is important to estimate power

• It is very important to consider power while designing an experiment.
• You should avoid spending a lot of time and/or money on an experiment that has little chance of finding a significant effect.

Questions

1 Power is:

	The probability that the null hypothesis is true.
	The probability that the null hypothesis is false.
	The The probability a false null hypothesis will be rejected.
	The The probability a true null hypothesis will be rejected

Answer >>

It is the probability of correctly rejecting a false null hypothesis.

2 If the power of an experiment is low then

	The experiment will likely be inconclusive.
	Any significant findings obtained are suspect.
	The results are skewed

Answer >>

With low power, the null hypothesis is unlikely to be rejected. When the null hypothesis is not rejected, the experiment is inconclusive.

Power | Example Calculations >