Common A/B Testing Pitfalls — Roadmap Step & Resources
Peeking at results early, novelty effects and other ways experiments go wrong
- Level: advanced
- Estimated time: 1 days
- Roadmap: Data Science
Before this step
Interview questions
- Why is 'peeking' at A/B test results early and stopping when significant a statistical trap?
Study resources
- Twyman's Law and A/B Testing Pitfalls (Article) — Microsoft's Experimentation Platform research group's articles on common A/B testing pitfalls.
- Common A/B Testing Mistakes Explained (Video) — Covers peeking, novelty effects and other common experimentation pitfalls.
Part of
- Experimentation & A/B Testing — section
- Data Science roadmap — the full learning path