Understanding the statistical power? or “sensitivity?” of a test is hong kong phone number data an essential part of planning before A/B testing. It will help you implement more changes to your site to increase revenue.
What is statistical power
Statistical power is the probability of finding significant results if a particular effect actually exists. It allows one to detect differences between test variants when they actually exist.
Before we move on to the components of statistical power? it is important to understand what errors there are and how to avoid them.
Two types of errors
Type I errors
A Type I error is a false positive: it rejects the null hypothesis when it is actually true .
The null hypothesis is the statement that there is no difference or effect between two events or phenomena.
In simple terms? the test shows this leads to the concept of authentic learning that there is a difference between the variants? although in reality there is no difference. The discrepancy occurs because the test is out of control due to errors or accidents.
The probability of a Type I error? denoted by the Greek letter alpha (α)? refers to the level that is already significant for an A/B test . If a test has a 95% confidence level? that means the remaining 5% is the probability of a Type I error (1.0 – 0.95 = 0.05).
If 5% is too high? you can reduce the probability of a false positive by increasing the confidence level to 99% or even higher. In this case? the probability of a Type I error will decrease from 5% to 1%. But such a decrease in probability carries certain risks.
Increasing the confidence level increases the united states business directory possibility of a Type II error. There is an inverse relationship between the alpha and beta errors: as one error decreases? the other increases? and vice versa.
The critical zone becomes smaller? and the smaller it is? the lower the probability of rejecting the null hypothesis? and therefore the lower the power level. It follows that if you need more power? you can? as an option? increase the risk of alpha errors (e.g. from 5% to 10%).