Statistical Power

The yellow curve shows the distribution of the sample mean X̄ under the null hypothesis H₀: μ < 0. The purple curve shows the true distribution of the sample mean X̄ which is centered around the true unknown μ which can be set in the slider on the left. In reality we don’t know this value, but we work under the assumption in this app that the true μ is not equal to the hypothesised μ (i.e. the μ slider does not go down to zero).

Both distributions have a spread defined by the SEM, σ/√n, as we discussed in the section on the central limit theorem. Both σ and n can be set on the sliders.

A “positive” result means to reject the H₀, a “negative” result means to not reject the H₀.

The dashed line is the critical value of the sample mean beyond which we reject H₀ with a confidence of 1-α. By choosing α, we fix the type I error rate (i.e. false positives) we are willing to accept (i.e. α).

The type II error rate (i.e. false negatives) resulting from a given critical value depends on the unknown μ (the center of the purple curve). If μ is far away from your hypothesised value H₀ (the center of the yellow curve), the true underlying distribution of X̄ doesn’t overlap with the null distribution very much, so the probability of observing x̄ lower than the critical value is low. That is, we have a very low chance of accidentally not rejecting H₀.