This article originates from a presentation at the Revenue Marketing Summit in New York, 2022. Catch up on this presentation, and others, using our OnDemand service. For more exclusive content, visit your membership dashboard.
My name is Milan Coleman and I work at LinkedIn, doing testing and optimization programs for LinkedIn ads. Today, I want to talk about my passion: experimentation.
People don't often think of experimentation as exciting, but I think it's a goldmine for marketers and it's super fun. Plus, it gives you the opportunity to learn so much about your customer through quantitative and qualitative research.
In this article, I want to explore a few questions:
- Why experiment, and what can stand in the way?
Hopefully, you already have a culture of experimentation at your company, but you may still find yourself coming up against certain challenges when you're setting up experimentation programs. - How do we define success and get leadership support for our experiments?
Honestly, this is something I've struggled with throughout my entire career.
Attribution’s obviously an issue for marketers, and so is making leadership understand what success looks like from a testing point of view – it can be hard to translate success from a test into a dollar amount. - How can we establish a system for research, iteration, and sharing past wins and losses?
When you have a large data set, it's hard to keep track of what is and isn’t performing well. Plus, a bunch of different teams might be involved, which can lead to duplicate experiments that run into thousands if not millions of dollars.
To avoid this, you want to stay up to date on what other teams are working on and share your research.
Our title, the $60 million tweak comes from Obama’s 2008 presidential campaign. Optimizely, a company that runs A/B tests, was working on the email campaign and landing page, so they ran multivariate tests on the media and the call-to-action (CTA).
These tests revealed some fascinating insights about bias as well as why experimentation is so important.
For one, they tested video versus image. I'm very biased towards video; it tends to get a lot more engagement so I always assume it’ll win, as did the team working on this campaign.
However, in this test, video was the worst-performing variant. If they hadn't run a test, they might have missed out on $60 million of funding. That's why it's important to validate your ideas with your customer to see what resonates with them; often, it's not what you think it’ll be.
As I mentioned, they also tested the CTA. They tried “Sign up” and “Learn more.” This seems like a small change, but it's about evoking an emotional response in your customer and tapping into what they want.
They don't necessarily want to sign up; they just want to learn more, so that was the winning variant. I really love this test because it's so powerful in showing the power of testing things out and seeing how your customer feels.
Why experiment, and what can stand in the way?
Before we get into the how, let’s look at the why. What are the benefits of creating a culture of experimentation?
Firstly, it provides you with direct ongoing quantitative and qualitative data about what your customers desire. With each experiment, you understand your customer more and more.
Secondly, it's really fun to test out new channels, formats, and strategies. Something we're seeing a lot of success with at LinkedIn is running an AI chatbot.
It’s really interesting to see the impact of new channels, especially if you're an early adopter and you start testing before everyone else jumps on the bandwagon.
If you experiment well, you can increase your revenue quite a bit and spark innovation not just with your marketing, but with your product.
At LinkedIn, we're doing a lot of marketing tests and they’re informing our product roadmap. We’re taking our wins to the product team and demonstrating what customers want to see, then aligning on ways we can work together and improve the product.
We’ve seen some of the benefits of testing, now let’s talk about some possible causes of testing fiascos – we also see those quite a bit in the industry.
- Inability to prioritize ideas: I’d say this is the biggest one. It’s great to have everyone bringing their ideas to the table, but if you don't have a way to prioritize them, you’ve got a problem. You can't do everything at once, so you need to understand how to prioritize.
- Non-subject matter experts involved in the project: It’s okay for them to be involved as long as you establish who’s an expert. Everyone having an equal seat at the table isn’t necessarily a good thing if they don't understand, say, certain aspects of design or tech.
- Attribution errors: This keeps me up at night.
- Unreliable datasets: If you're looking at Google Analytics and it's just not lining up with your other metrics, you’ve got a problem.
- KPIs that aren’t tied to business goals: As web marketers, our KPIs are usually about click-through rates or the time spent on a page, but executives might not understand how that translates into dollar amounts.
- Duplicate tests: You need a way to share information so everyone can reference it and not keep doing the same things over and over again. This is so important because it takes a lot of time to set experiments up and they can get really expensive as well.
How do we define success and get leadership support for our experiments?
It’s crucial to create something that not only your customers like but also something that resonates with leadership – you can't move forward if leadership doesn't buy in.
To make this happen, you need a joint effort from both your internal team and your external customer team.
