1. Are A/B tests important?
The goal of every website, store, or application is to tailor its design and functionality to the needs of its users, to provide them with the best possible user experience. Elements of graphic design such as colour schemes, buttons and pop-ups are intended to encourage users to take specific actions that are important to us, for instance subscribing to a newsletter, buying discounted or abandoned items in the shopping cart. Although we strive to improve our design to best suit our target audience, we can’t predict how every detail will be perceived by users. Often, even a small thing like a banner type or colour of the button can determine if a user performs a specific conversion. How do we decide what works best for users of our website or application? A/B tests can considerably help us to understand real preferences of users.
2. What are A/B tests?
A/B tests are experiments that involve comparing two versions of an application, website, or online store. Variant A (control) is the main version, while variant B (modification) is the test version including the change we want to introduce. During A/B testing, randomly selected users are assigned to one of the variants. Then, after analysing users' reactions to each variant, we know which one is more effective.
For example, if we plan to offer a discount in our online store and want to inform users about it through a pop-up, we can prepare two versions of the pop-up with different colour schemes, graphics, layout, or font styles. The change can be minimal, such as just the button colour. With the help of appropriate tools that track user behaviour, depending on the variant of our pop-up, we can tell which one was clicked more frequently by users.
3. How to prepare for A/B testing?
A/B testing primarily aims to examine users' preferences on our website or application. This information is incredibly valuable in increasing sales effectiveness and building user engagement. The better we prepare for A/B testing, the greater the chance of achieving a specific goal. Below we have described the steps that we should take to properly prepare for A/B testing:
- Step 1: Define the goal of the test
- Step 2: Select the element to test
- Step 3: Create two variants
- Step 4: Divide users into groups
- Step 5: Monitor user behaviour
- Step 6: Draw and compare conclusions
It is essential to determine the outcome you want to achieve through A/B testing. Consider whether you want to increase the number of conversions, improve the click-through rate (CTR) of a specific element, enhance the readability of a section, etc. This will make it easier for you to prepare variants for achieving this goal.
Depending on the defined goal, choose the element that will have the biggest impact on its achievement. It can be a single element or section, as well as an entire subpage, such as the homepage or product page.
The next step is to create two versions of the selected section, element, or subpage. Variant A and variant B can be completely different from each other or have relatively minor differences. When preparing the designs, remember to maintain consistency with the existing visual identity. If you want to test user reactions to big changes, test them gradually to avoid disorienting users with completely different view of the subpage.
Users should be randomly assigned to each group. If we want to obtaining even and reliable results, we should divide the group of users of our application or website into two equal groups. Depending on the type of test, decide how many people should participate in the test.
While running the test it is extremely important to monitor users behaviour in both groups. Thanks to such analysis we can collect essential data such as conversion rate, time spent on the website, the number of clicks, the path that users followed, etc. The duration of tests depend on many factors such as website traffic or the number of tested changes. You need to decide what amount of information will confirm the need of making changes.
The information collected during the test is valuable for further optimisation of our website, online store, or application. Based on this information, we know exactly what works best for our target audience and in which direction we should continue the optimisation. You need to determine which data is most relevant for the conducted test and what value it should achieve to consider a particular variant more effective. It is also crucial to determine the proper level of statistical significance that points out what should be the difference between the score of each variation to assume the result as relevant. The most often statistical significance score is between 95% and 99%.
4. A/B testing tools:
Below, we have described some solutions that can be used to implement A/B testing in your project. Before choosing a specific solution, check its integration possibilities with your online store, website, or application.
1. This solution involves adding a script that creates a random division of users and assigns them to A and B variants that we create. In this case, user behaviour can be tracked using integration with analytics tools such as Google Analytics (which will require additional configuration). Although this solution seems relatively easy to implement, it may be insufficient depending on the scale of the changes we want to introduce and the data needed for analysis.
2. Integrating our website or application with a dedicated A/B testing tool is an easy way to add functional tools that support our A/B testing. Dedicated A/B testing tools include many useful functionalities that facilitate and accelerate the testing process. For example you can use:
- Optimizely - It is one of the most popular A/B testing tools. It allows for easy creation of variants and monitoring them from the panel. It also offers more advanced features for content personalisation based on different user groups (users can be grouped by age, location, etc.). It also has the option of multilateral testing (testing more than two variants).
- VWO (Visual Website Optimizer) - Another tool that allows us to conduct A/B tests and personalise the website based on user behaviour. VWO offers a free starter option for those who want to start their journey with A/B testing. It is a great way to see how this tool works and how it can help us.
- Crazy Egg It is a tool for comprehensive analysis of user behaviour, including the ability to conduct A/B tests. Crazy Egg allows for creating heat maps that show areas of the website that receive the most attention from users. It also offers recordings, which provides detailed insights into user behaviour on our website.
- Adobe Target This solution is part of the Adobe Experience Cloud package. In addition to advanced tools for analysing user behaviour, Adobe Target allows for dynamic website personalisation based on user preferences (more on this topic in our article on Marketing Automation).
The above-mentioned tools offer a similar range of functionality. Firstly, when choosing an A/B testing solution, consider your expectations. Do you need advanced user behaviour analysis options or an intuitive platform for quickly preparing variant projects? Or maybe you're looking for something that combines both features? Another aspect worth verifying is the integration capability with your store, website, or application. Of course you will also take a look at the price of a particular solution, However this factor is usually individually estimated based on selected features and project-specific requirements.
4. A/B Tests for e-commerce websites
A/B tests are very crucial for the e-commerce sector, which has to dynamically react for upcoming changes, especially in terms of UX and UI. Market research helps us gain knowledge about users preferences, but they won’t provide the details about individual customers who are part of our targeted audience. A/B tests have a real impact on improving customer experience as we examine real reactions of individual users. Based on this we can effectively optimise the design, layout, colour scheme or content of our online store and see how this would affect our website or application and the number of conversions.