Split Testing

Before you begin, it might be helpful to review some guidelines for effective A/B testing.

1. Only conduct one test at a time - If you test an email campaign that directs to a landing page at the same time that you’re A/B testing that landing page, your results can get muddled pretty easily. How would you know which change caused the increase in leads?
2. Test one variable at a time - In order to evaluate how effective an element on your page is, you have to isolate that variable in your A/B test. Only test one element at a time. 
3. Test minor changes, too - Although it’s reasonable to think that big, sweeping changes can increase your click numbers, the small details are often just as important. While creating your tests, remember that even a simple change, say, switching the image in your email, can drive big improvements. In fact, these sorts of changes are usually easier to measure than the bigger ones.
4. You can A/B test the entire element - While you can certainly test a button color or a background shade, you should also consider making your entire email a variable. Instead of testing single design elements, such as headlines, subject lines and images, design two completely different emails and test them against each other. Now you’re working on a higher level. This type of testing yields the biggest improvements, so consider starting with it before you continue your optimization with smaller tweaks.
5. Measure as far down the funnel as possible - Sure, your A/B test might have a positive impact on your email click rate, but how about your sales numbers? A/B testing can have a significant effect on your bottom line. You may even see that an email with fewer click produced more sales. As you create your A/B test, consider how it affects metrics such as traffic-to-lead conversion rates, and demo requests.
6. Set up control & treatment - In any experiment, you need to keep a version of the original element you’re testing. when conducting A/B tests, set up your unaltered version as your “control,” the email you would normally use. from there, build variations, or “treatments,” the email you’ll test against your control. For example, if you are wondering whether including a testimonial in an email would make a difference, set up your control email with no testimonials. Then create your variation with the testimonial.
7. Decide what you want to test - As you optimize your email, there are a number of variables you can test. You don’t have to limit yourself to testing only images or text size. Look at the various elements on your marketing resources and their possible alternatives for design, wording, layout. In fact, some of the areas you can test might not be instantly recognizable. for instance, you can test different subject lines, senders, or ways to personalize the message.
8. Split your sample group randomly - In order to achieve conclusive results, you need to test with two or more audiences that are equal. With HubSpot, we automatically split traffic to your variations so that each variation gets a random sampling of visitors. You can decide the size of the sample group using the slider in Step 5.
9. Test at the same time - Timing plays a significant role in your marketing campaign’s results – be it time of day, day of the week, or month of the year. If you were to run test A during one month and test B a month later, you wouldn’t know whether the changed response rate was a result of the different template or the different month. A/B testing requires you to run the two variations (A and B) at the same time. Without simultaneous testing, you may be left second-guessing your results.
10. Decide on necessary significance before testing - Before you launch your test, think about how significant your results should be in order for you to decide that the change should be made to your website or email campaign. You can choose a winning email version based on opens by delivered, clicks by delivered, and clicks by opens.