Statistical Power

The yellow curve shows the distribution of the sample mean X̄ under the null hypothesis H0: μ < 0. The purple curve shows the true distribution of the sample mean X̄ which is centered around the true unknown μ which can be set in the slider on the left. In reality we don’t know this value, but we work under the assumption in this app that the true μ is not equal to the hypothesised μ (i.e. the μ slider does not go down to zero).

Both distributions have a spread defined by the SEM, σ/√n, as we discussed in the section on the central limit theorem. Both σ and n can be set on the sliders.

A “positive” result means to reject the H0, a “negative” result means to not reject the H0.

The dashed line is the critical value of the sample mean beyond which we reject H0 with a confidence of 1-α. By choosing α, we fix the type I error rate (i.e. false positives) we are willing to accept (i.e. α).

The type II error rate (i.e. false negatives) resulting from a given critical value depends on the unknown μ (the center of the purple curve). If μ is far away from your hypothesised value H0 (the center of the yellow curve), the true underlying distribution of X̄ doesn’t overlap with the null distribution very much, so the probability of observing x̄ lower than the critical value is low. That is, we have a very low chance of accidentally not rejecting H0.