“To keep this article as simple as possible, I intentionally avoided the issue of statistical significance. This can get real complicated real quick, but it’s basically a way to prove that the results of an SEO test are reliable and that there’s a high probability of a causal relationship between the change and outcome. In other words—that whatever happened (or didn’t happen) is unlikely to have been by chance.
I’m not currently aware of any SEO testing tool that calculates this for you. So let me reiterate my earlier advice: don’t run SEO tests unless your website gets plenty of traffic.
If you’re running tests and making conclusions after only a few hundred visits, chances are the results aren’t going to be statistically significant.”