A/B Testing for Product Managers
A/B testing, or ‘split-testing,’ is a method of testing variations of components on web pages, app interfaces, ads, or emails. Designers or marketers start with a goal of, say, earning more sign-ups and test two versions with similar audiences. If there’s a clear winner, they adopt that version.
What are the benefits of A/B testing?
A/B testing injects a healthy dose of reality into your products. It’s all too easy for product designers, developers, and marketers to fall in love with their own creations and A/B testing shows them what users really think.
For example, when designing a website, how can you determine the optimal location for each button? Or which graphics maximize sign-ups? Or what version of your navigation bar do users prefer? Only A/B testing can tell for certain.
Here are some benefits of A/B testing your digital product:
- Optimise performance
- Reveal hidden trends and user behaviors
- Incorporate user feedback
Testing takes you right to the heart of users’ desires for your product. More often than not, you and your users share a goal: you want to help them solve a problem, enjoy an experience, make a purchase, or get help.
By testing and optimizing each stage of their journey, you can speed up their travels and both get more of what they want. Plus, A/B testing is non-invasive and doesn’t disrupt their experience because most of it occurs without their active knowledge.
After repeated tests, the product slowly evolves into something that’s intuitive, useful, and which people love.
When can you use A/B Tests?
Here are some common examples of A/B testing goals for several verticals:
- E-commerce site: Increase purchases
- Consumer app: Increase time spent in-app
- Online publication: Increase sign-up conversion rates
- Business app: Increase revenue-per-user
- Advertising: Increase clicks and purchases
- Email marketing: Increase opens and click-through rates
Now, to achieve these goals, the product or marketing team might want to test different variation of components like:
- Layout
- Navigation
- Buttons
- Text
- CTA
- Social proof
- Pop-ups
- Announcement bars
- Widgets
- Badges
- Images
- Videos
- Links
These elements can be tested in terms of their size, placement, color, or even the timing and circumstances of when they appear.
How can you perform an A/B Test?
Step 1: Define your goal
Step 2: Look at your current data
Step 3: Come up with a hypothesis
Step 4: Perform A/B Test and analyze the results
Step 5: Repeat with something else
Now, let’s look at an example:
Let’s say you are working at an e-commerce company and you want to increase the clicks on the buy button. Now, as a PM, you are supposed to come up with an idea.
Step 1: Increase the clicks on the buy button on product pages
Step 2: You go through the data and find that current click rate on product page is 20% but the buy button is too small for the user to notice.
Step 3: You come up with a hypothesis “Making the CTA button more prominent will drive 20 percent more clicks”
Step 4: You increase the size of the buy button only for 20% of your users and let it be the same for 80% of the users. Then you observe the numbers over the next 1 week.
After 1 week, you found that for the 20% users, the click rate was 21%, while it was the same 20% for the users with the same button.
Although the click rate is higher, but the change is only 5%, way below our set hypothesis, so we won’t implement the change.
Step 5: Implement any learnings and continue to test. Each time, return to your conversion funnel and use A/B testing to identify and remove blockages to users’ progress. In doing so, you’ll increase the utility of site or app.
Do you want to become a PM?
For starters, you can follow me, go through the other stuff I have written, and learn as much as you can from here.
If that isn’t enough or doesn’t help, you can checkout this toolkit I created for beginners like you to get started with PM topics, learn the skills they really need to become PMs and prepare for PM roles.
If you want a more personalised learning experience, consider joining our Product Management cohort at Crework — Apply now
