You know that A/B testing is an important part of creating marketing campaigns that work.
But did you know you can create worthless A/B tests? Here are 5 types:
- The Minute Markup: This test compares such tiny differences that they aren’t likely to be impactful, much less distinguishable. (e.g. buttons with a 10 pt border vs. a 5 pt one).
- The Self-Fulfilling Prophesy: This test is designed so that the “winner” can only be the variable that was pre-selected. This happens in academic research all the time, where the test is structured so that there is no other possible outcome. In this case, the results are true, but also misleading.
- The Convoluted Comparison: This is a test where multiple variables are tested concurrently. It’s important to only test one variable at a time. If you test multiple variables, you won’t know which influenced the change.
- The Time-Waster Test: This test produces results that you don’t have the capacity to act on. For example, you might run a test that discovers that a small proportion of your audience responds best to CTA language that differs from the bulk of your audience. While this can be great information to have, if you don’t have the resource to build custom campaigns for that smaller group, there’s no point in getting results you won’t actually use to market better. Strategically prioritizing marketing tactics can help you know what’s worthwhile to test.
- The Statistically Suspicious: This test has such a small recipient audience that it isn’t statistically significant. A good rule of thumb for marketing A/B tests is that you want a list size of at least 1,000. If your database is smaller than this, consider reviewing other test results to make better informed decisions.
Need help crafting tests that are reasonable, actionable, and produce marketing that works? Reach out! I’ll pair my academic and business research experiences to help you deploy tests that improve your bottom line.