What confidence level is: When you are calculating statistical significance, you need to decide what you want your confidence level to be. A confidence level tells you how sure you can be about your results. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain.
What confidence levels are not: a margin of error. The confidence level is how often that percentage of the population (say 95% or 99%) who would pick an answer lies within the margin of error– so the margin of error does factor in, but it’s not the confidence level.
Why we care about confidence levels: We need to know what confidence level we’re comfortable with when calculating statistical significance. Need to calculate your statistical significance? Try our statistical significance calculator.
This post is part of an ongoing Clickipedia series, defining all the terms you need to know for conversion rate optimization. Keep checking back each week to learn more, and don’t forget to check out the whole series.