Marketing is simple and digital marketing is simpler.  All you have to do in order to be successful is run some A/B tests and you’ll know exactly what needs to be done to scale.  Not so fast.

Conversion rate optimization is a hot new trend for companies focused on growing their web revenue, making customers stickier, and scaling quickly.  What may surprise you is CRO is more entailed than changing a button color, adding better sales copy, or changing photos – great conversion rate optimization is built on a solid foundation and culture.

We sit down with Guido Jansen to get into what really matters for great CRO.  Guido shares the four pillars of conversion rate optimization, where the journey to success really starts, and how to build a team passionate about finding the best outcomes.

Kyle Hamer  9:43
Welcome back to another episode of summit podcast. I’m your host Kyle Hamer. Today’s guest on the show is Guido Jansen from the Netherlands. Hito Welcome to the show.

Guido Jansen  9:53
Thank you for having me.

Kyle Hamer  9:54
I hope I got that right like was that that was a pretty good not too bad for butchering it for an American.

Guido Jansen  9:58
Alright, great.

Kyle Hamer  10:01
For those of you who don’t know, Hito Hito is a conversion rate optimization specialist he has more experience than you can shake a stick out when it comes to understanding how to do e commerce, convert sites and sell things online. And if you really break it down, the reason he’s good at what he does is because he’s got a background in cosmic cognitive psychology. And he uses that way people think and the way people buy in turns it into experiences that convert, does that sound about like you?

Guido Jansen  10:31
Yeah, that sounds about right. And, and to be clear, the backgrounds in psychology doesn’t allow me to, to read people’s minds or something that’s, that’s, I sometimes need to explain that on parties. But it does allow you to have a solid background in basically how people make mistakes, when they make decisions with the kind of mistakes we make the limitations of our brain and how we can navigate that.

Kyle Hamer  10:59
Well, you know, it’s it’s interesting that a lot of marketing boils down to psychology. And that’s part of the stuff that most people miss in the one on one stuff, right?

Unknown Speaker  11:07
Yeah, actually. So when I started doing psychology, that was all nice. And I started building websites during, during my time in college already, and I started doing stuff in it right after. But it makes my mom very happy to still do something with my background in psychology. But yeah, in the end, you still, despite this, being through the internet’s and through a computer screen, you’re still trying to sell something. And if it’s, whether it’s a product or service or an ID, you’re trying to sell something through that webpage, and there will be hopefully, and then for the foreseeable future, there will be a human on the other side,

Kyle Hamer  11:44
for the receivable future, at least. Now, from a technical standpoint, you have a lot of experience when it comes to Magento. You host a podcast called crmo. cafe, right? Isn’t that’s not that’s what it’s called?

Guido Jansen  11:57
cafe. Okay, CRA

Kyle Hamer  11:58
cafe, you host Magento events there, you you’re doing speaking internationally. So it’s not like this is your first rodeo, it’s like we’re Hey, you know, I just learned out that I need to change this color to that color. And that makes you a specialist. There’s some there’s some experience here and what you’ve been doing.

Unknown Speaker  12:16
Yeah, commerce can make a difference. As always, the lame example that we use it if we want to see rose wants to talk about a bad AB test we just did. The example is always going to be the the call to action button that you’re going to change color to red, or green or orange or whatever. I mean, if it’s if it’s really bad design, it can make a difference, but probably not your biggest bottleneck. Now.

Kyle Hamer  12:44
If you haven’t guessed by now, as you’re listening in, we’re going to talk about conversion rate optimization today. And to kind of set the stage you know, I think it’s always really good to have our definitions aligned so that we’re talking about the same thing. When we talk about conversion rate optimization, what are we talking about weed? Oh,

Unknown Speaker  13:01
yeah, that’s that’s a question CEOs often ask themselves, is actually, that I’m doing or should be doing. Because if you just look at the term, zero, conversion rate optimisation, it’s a very narrowly defined work area, right? You You’re there to optimize the conversion rates. But actually, if you just think about it for a couple of seconds, then you will know that’s not probably the best, or at least not the only metric it’s you want to optimize. And it wouldn’t be the first time when I that I get to two clients where they say, okay, we want you to do just to optimize the conversion rate, it’s the only thing you need to do. Are you sure because and the silly example, is that, okay, but I can just block off of your worst performing traffic. For example, if you have affiliate traffic, that’s probably performing worse than organic traffic, if I just block all the affiliate traffic, your conversion rate will go up, easily, I can double it. But you know, it’s gonna make more money, it’s gonna hurt you. Your revenue, it’s gonna hurt your profits, probably. So that’s the conversion rate alone is not what you’re after. So basically, what the the, what we are doing as heroes, what the people have been doing this for a long time. Basically, we are an often this is applied to website. But as a side note, it doesn’t only need to be applied to websites or digital interface can be anywhere. But basically what we are trying to do, we are the detective that find out where users are getting stuck. That’s one important parts of where they’re getting stuck, and figuring out how to solve that. And the second part is okay, we figured out all the places that we made a very smooth ride to the answer. It’s very easy to do something. And the second part is how can we make people actually want this Are there any personal Rates of blockages out there that people actually rather go to, to another website to competitor than on your website, how can we persuade them to buy something at your website. And so and then there are a lot of ways of being this detective, you can you can look at Google Analytics, you can look at tools like old jar, or usabilla, do surveys on your website, you can actually sit down with people do user research. those are those are often methods of figuring out where people are getting stuck. And what we can do about this. Of course, a background in psychology helps to interpret that data. Because while you have all this qualitative and quantitative data, and you need to extract some meaning from that, for example, Google Analytics data is great at telling you where people are getting stuck, but you have no idea why. If you’re interviewing people, that’s very time consuming. Maybe on a day you interview like six people. So what that’s gonna mean, and in the grand scheme of things, there are issues with all these research methods. But combining those that’s where the power I think, is with good zero people combining all these different data points and research methods, combining that into a current view. Okay, where are people getting stuck? Why are they getting stuck? And what are we going to do about it? And then the next step is validating these hypotheses. That’s what people often think about when they think about zero is running an A B test. And that’s, that’s indeed one of the things you can you can one of the methods you can use to to validate whatever hypothesis you have.

Kyle Hamer  16:40
You know, it’s interesting that you bring up A/B tests, because there are a lot of there are a lot of marketing organizations, or even companies in general who lean heavily on the internet is their primary source of income, right? It’s, it’s the only way they’re making revenue. And if there’s, if there’s a blockage, there’s these these obstructions, right? If there’s these places where people stop, this means they’re losing money. However, the answer always tends to be just run an A B test. And you’ve got some insight in that, you know, A/B tests are really not as successful as people would think, right, that these experiments and things that you run, they’re not, they’re not successful. So there’s almost as much as people are leaning into, you know, the e commerce and driving Revenue Online, there’s also going to kind of be a culture shift. Can you tell us what, what really makes up a good culture for successful sorrow and successful optimization for an organization?

Unknown Speaker  17:34
Yeah, so it wouldn’t be the first time then when you go into a company as a CEO that that people? Well, first of all, the first week, everything, every thing is all nice. And people think it’s nice, it’s sexy to run experiments somehow. Which is, which is nice. It’s beneficial for, for people doing so zero. It’s unsexy, to run experiments and to be very data driven. But then, from that point onwards, basically what you’re doing, or how people often can perceive this is, okay, I’m an I’m the content editor on the website and the campaign manager, I’m the designer of that website, I’m the developer of the website, I created this, the whole websites. Basically, you’re coming in and saying, Hey, this is wrong, this is wrong, this is wrong, this can be better. We need to optimize that. So basically, your it can feel like you’re critiquing everyone that’s been working there, all the campaigns that they run everything that they have there. So that you need to be able to manage that as a zero specialist. And then it helps when there’s a culture in the in the company. That that’s okay with people not knowing and not failing. And especially when you’ve done zero, when you’ve run a couple of 100 a beetus. You get very humble. You know, okay, my, my opinion doesn’t really, really matter in, in improving our websites, we should really validate stuff. And just to give you a number are the big companies out there that this is a big example of continuous optimization through running a little experiment. That’s the Dutch originally Dutch company booking.com. They have just the numbers are probably different because they’re quite secretive, but some numbers that they put out there publicly, that have around 75 product teams doing this, maybe maybe less, bit less this year, but 75 product teams all running. The only thing they put something live on the website is through an experiment. They’re in 1000s of experiments. But from all those experiments they they put out there and as it reminded this is this is their bread and butter. This is what they do. They run experiments is basically what the company is about. And they happen to sell hotel rooms. But that’s their core business running experiments be really good at doing that research. And still, if, if you’re in that mode, if you’re that good at doing this, nine out of 10 experiments that booking.com puts out there are basically while failure in the sense that it’s not going to be implemented, it’s not that there’s not moving the numbers in the way they want it to move. And there’s people doing this for a living. So you can imagine that if you’re, if you’re your team, your content team, if you’re not running experiments, and just doing everything based on gut feeling, or whatever, the highest Highest Paid person in the room says or thinks that number is going to be worse. That’s not good. So you need a culture in your company that allows people to be comfortable with that. And that’s not something that’s going to change from one day to the other. And it’s something that you need to build upon. And zeros can definitely help that. Definitely whenever when I first started with with a client, I would always suggest at least spend 50% of your time not running those experiments, doing that research, but spend it on the internal company culture, try to figure out what what makes people move, try to figure out what the actual KPIs are, that they’re working on? Well, the one of the first talks I always have is with the BI or finance team, what are the numbers they’re actually looking at. So we can make sure that’s when I RS heroes mission, especially as I started a company that we are sure that we, we we are here to help those people, right that built the website, we’re not here to critique your, your, your, your job, your work, we’re here to make this make your job align better with whatever it is that the user needs or wants at that moment. And of course, also, including alignment with the business values that we have, and the business metrics that we want to improve.

Kyle Hamer  22:02
Well, and so it sounds like I mean, it sounds like some of the experiments that you’ve you’ve seen with booking.com aligns very similar to what we hear with, you know, the big the big behemoth here in the States amazon.com. They’re pushing. I think it’s between 11 and 20, new experiments or tests every minute, right? It’s not, it’s not an hour, like they’re, they’re just they become this test, test, test test, whether it’s an experiment on pricing, or an experiment on, you know, where a buttons placed, or how a review is at or like they’re always constantly experimenting. There’s definitely a cultural element to that. And that sounds like to me that it really starts with looking at the business objectives and and aligning failure to equate to success against those business objectives. How do you how do you resolve that?

Unknown Speaker  22:52
Yeah, so you need to be comfortable with that, as you’re you’re doing this that? You don’t know that I know how to find out is this euro specialist, but I don’t know, that’s certainly what the answer is and the beginning of my career, I vividly remember a client’s saying, Okay, yeah, we want the they wanted the NEW Webshop on Magento. And they said, we want we want we really like this, this website from this, this company. Here in the Netherlands, we like their style, we should we should copy that for our websites. And like, they are selling they are selling clothes, you’re selling bicycles. You have you know what to do, right, your, your, your psychologist or zero, you know how people think so you know, what, to what to do and how to make this work, right. But I’m not your customer to start with, I’m not specialized in cycling websites, per se. But yeah, we can find, of course, we can find it out, we can, we can invite people to do that. And figure that out. So that’s basically how zero zeros work. And the idea is that we can try to we try to spread that in an organization that’s not always successful, definitely the organization needs to be open to it, those KPIs need to be need to be aligned. It’s very often happens that’s, and it’s also hard. It’s not necessarily an easy thing to do is not surprising, that’s company, but the ultimate goal of every company should be something like lifetime value. That’s not something I can measure it in an experiment. I mean, an experiment usually runs like maybe four weeks, on websites, maybe two weeks. That’s usually the bandwidth two to four weeks for an experiment. You’re not going to measure lifetime value, but because it’s something you only know after a couple of years, maybe, but you need to figure out what are the indicators there. are the indicators afforded lifetime value? Can we go for like a restart instead of the beginning, pure conversion rates is the other end of the spectrum, this book probably also, you can measure that directly, but it’s probably not the best metric to use. So you need to figure out maybe its average order value or revenue per user, something like that, that you need to figure out. For example, I want to start working at a company a couple of years ago, the first thing that’s ecommerce managers, that’s like 12 ecommerce managers for for several countries. One country equals measure for each country. And basically, the the reports they filled every day in the morning was, and it was a very detailed game from the br department report based on the order numbers that they got in yesterday, compared to what they expected to get in. And so they had a dislike rolling forecast for 365 days. Based on all the investments they did on, it was a gift driven, business over holiday driven. So when all the days were when they were investing in some countries or not what the competitors were doing, it all elaborates forecast on what the numbers should be what they expect. And even not only pro websites, or natural numbers, or numbers per website, but even per channel on the websites, they’d had this prediction, all very nice. But again, it’s ordered numbers. And that’s it was the highest, or the best number they got at that time. But if that’s what you what you look at every day, order numbers need to go up, that’s the best number you’ve got to deal with everyone in the company is looking at, that’s what your manager is talking about. That’s what you’re talking about, during your annual review that those are really important reports that are most important. Of course, you start steering on those and and those are smart people they notice is not the best metric, because everyone just started to give away discounts. That’s what everyone was focusing on running promotions, running discounts. Because Yeah, the order numbers will go up. But of course, again, like like pure conversion rate, that’s not the best metrics or serum. Because they basically they didn’t have any profit

Kyle Hamer  27:23
whatsoever in, in my limited experience in working with e commerce, when you start rolling out the discounts, you’re one trading your audience or your customers to wait until you provide a discount. So that’s going you know, they’re only going to buy when you’re decreasing your margin, that’s just the way that they behave. So either you jack up the prices, and you don’t look competitive, so that you can still give them a normal margin, or you’re, you’re really living off of, Hey, I wouldn’t really make between eight and 12% on this particular transaction. And now I’m only going to make between two and 3%. And that really changes the, you know, the macro economics of the company. So it sounds to me like what you’re saying is, is that, you know, when you’re looking at creating this culture, starting at the business objectives, and in focusing in on what’s really important to the business of how they, how they view what’s important to them, whether it’s, you know, it’s it’s speed, or volume, or, you know, time to convert or average transaction, it’s really intimate. It’s like there’s no one right way to do conversion rate optimisation. Is that a fair assessment?

Unknown Speaker  28:29
Yeah, it might sound sounds we’re hearing is for from from a CRM specialist, but the data is not the things that we can measure in an experiment is not everything. I mean, there are a lot of businesses out there that maybe if you if you have a premium brand, for example. And the brand image that you have is really important. That’s not something I can measure in two to four weeks. That impacts if I run a promotion, that’s something I can directly measure. And if the only thing I’m looking at is those promotion numbers in the short term, sure, the promotion is gonna win against not running a promotion at all, or doing some some brand specific things. But the power in zero is that we can measure that impact, we now know, what is the impact of doing A versus B, on the short term, and I’m fine with people then still going for the option that that makes less money in the short run if we have a vision behind Barbara doing this, but at least then we know, for example, I was working at a company and they implemented a completely new style on the website, new colors, new everything. And one of the things that well, the designers were very offline focused, let’s say to say that way. And they came up with the main call to action on the on the website that they created this in the new style was a Was it ghost, but if you don’t know, a ghost button is basically a button, that’s the same color as the background, with a thin white line or any color line with a thin line around it as a border. Well, from from my background already, now, if you if you study psychology, if you if the color is the same color as the background is not going to pop out, it’s not going to be the most visible element there, you need contrasting elements to pop out. Okay. And but if that’s the main, and we did some user studies with that people just didn’t see the button, they didn’t see it as a button. They didn’t, they were not inclined to click it. So conversion rate on that page, from that page to the next page went down drastically. And those are the things I mean, if I can bring that to the brand, even if they have good reason why this is really important for their brands. Sure, they can go with, with with a ghost button. But if that’s, if that’s if that’s a hits, you want to take in, in the short term to to benefit you. And if you think that’s benefit, beneficial to you in long run? Sure, now we can say, Okay, if you do this, this is fine. But it’s, it’s gonna cost you 50% of your revenue right now, this month, that’s the, that’s the impact of this new design is going to have, are you willing to take that risk. And, and this is helpful, and that’s authorized, see people struggling with with, for example, the brand theme that I’ve done a very strong feeling about the colors that you should use or the style that you should use? And I’m fine with having that they’re having a voice and having having an opinion on that. And brand is important. It really is. But now we can quantify this, okay? Do we are you willing, and you can just bring it to the product manager or whoever’s in charge of the website, say, okay, you can go either way, but this is gonna This is what the impact is on your on your bottom line, and are you comfortable with with doing that

Kyle Hamer  32:07
still comes back to psychology, right? I mean, whether its internal or external, you’re, you’re you’re, you’re manipulating or setting it up so that people are making decisions and empowered to make decisions, you just happen to be guiding them in a direction that hopefully is supportive of what the business wants for them,

Unknown Speaker  32:24
I really see zero as an informational role for for the product manager. That we are, we’re here to inform the product manager, or the product owner, and tell them, hey, these are the choices you want to make. And this is going to be the impact, the expected impact when you make those choices, and then again, up to you what choices you make, because like I said, there, there are quite some metrics still, that even with doing rigorous experimentation that we cannot measure, like, brand value, that’s not something we can directly impact and lifetime value is not something we can directly measure. And that’s fine.

Kyle Hamer  33:04
Now, we’ve talked a lot about culture in measurement. But those are really, in a lot of ways byproducts of our or at least a portion of the pillars of good CRL. Another pillar another strong component of CRM we’ve talked about, or at least I’ve, I’ve read is really your experimentation process, right? I mean, there’s the tools that you use, there’s how you implement them, and then ultimately, how you run experiments which drive the other two. So talk a little bit about what it means to experiment how you set up a good experiment. And in the things that you’re looking for. Sure.

Unknown Speaker  33:43
So just a walk through through the experimentation cycle that’s that we often use. In the beginning, I spoke about doing all this kind of research, try the Google medical jar surveys, interviews, whatever. These all come together and maybe they’re there’s there’s a manager in there that have had some opinions on how to change things fine. It’s just these are all these all come together in one big buckets of ideas and research findings. Then the idea is to start quantifying those based on the on the on the experience that you have and the priority system as you create a priority system. There’s many there are many priority systems out there to do this you can say okay, this this is based on research this is based on this expect it to to actually impact our main metrics or is it a secondary metric? How big is the impact? The estimated impacts are different development, for example, how long does it take to develop or something like this, those are all things that you can include in there. So you prioritize those. All those all those different ideas. And basically, then you can you get a testing backlog out of that you start on the top of the list of the things that you expect to have the biggest impact. Also you can, you can in your priority, you can also include something based on your own your experimentation history to saying, Okay, if we test on this specific target audience or specific channel, we expect, historically, we would expect better results, that’s fine, you can include it in your priority system. And then you start, you start building the test to create hypotheses, maybe you want to do some additional research. You define the test parameters. So what are the exact KPIs you want to test on the customer segments that you want to test on the pages, you want to test on the variants that you want to create a new borders that there is also to include some stakeholders. For example, you can imagine if I’m creating big experiments, right before Valentine’s Day or not a holiday that’s, that’s important to your company, you want to Well, at least you want to know about it. All the kinds of campaigns that you’re going to do, your your e commerce team probably wants to know about. And let’s see if this is indeed, okay moments to run the test needs to check that maybe there are some campaigns in there that you can even support with, with your experiments, you’re gonna deploy to test. When you when you set your parameters, you also calculate the required sample size, it’s there’s some statistics involved there, you need to know how many people do actually need to have in this test, to be able to say something about it. There are often requests coming in testing something, for example, on the on the order thank you page, it’s really hard to measure things at the end of the funnel, because you have low low traffic. They’re high conversion, but lower traffic. So it might be possible depending on the exact numbers, but you determine how long a test should run. And like I said, often between two and four weeks, after that’s, you run an analysis of the tests and see, see what happens. And then you share those learnings, that’s an important one, you want to share with whoever came up with with the ID and with your team with stakeholders share what the findings are, and then basically, rinse and repeat. Like you said that Amazon, they do multiple experiments, eat lunch each day. So the just the quality, the quantity of the tests, the number of tests you run is not everything. But it is important to have some frequency in there to have some regularity that you did you put out this regularly, in order to make this successful. Zero is not something you do on the side. You don’t just if you can only run one or two experiments a month, it’s probably not going to be worth all your all that efforts to just just do that, like if you if you would expect similar success rates as booking.com, nine and 10. And you have only one, one or two experiments a year, that’s going to be very depressing. So you need to be able to cycle I just mentioned, you need to be able to do this a couple of times a month, at least, to be able to success to be successful in this

Kyle Hamer  38:27
need to find success, meaning you talk about you know, looking at learnings and establishing your hypothesis. What happens if you you know, you complete your experiment, and you get through and you’re like, Well, it looks inconclusive. What do you do? Or how do you define success? And how do you define? Is this something that I do only during a short period of time? Or this is something that we implement forever? Like how do you? How do you like diagnose what the what the best way is to deploy this? Or to kill it?

Unknown Speaker  38:54
So do you mean the determining success of a single experiment or of a zero program is all

Kyle Hamer  39:00
it single experiment? So you know, the the attribution to your CRM process or your CRM program altogether? But a single experiment? How do you how do you know it’s not just an anomaly?

Unknown Speaker  39:12
Yep. So basically, there are four options. or roughly four options when, when you are for an experiment, either it’s a success as in it has an detectable effects on your, on your core metrics, and that’s in a positive direction for your for your company. It can be the other way around the detectable effect, but in the direction that you didn’t want to go. It can be inconclusive. And the fourth option is basically something went wrong with the test and run it again. That’s also not already Didn’t you didn’t reach the sample size and that’s also on like more technical problems with the test and success and it totally depends on the experiments. You can run different different times types of a beat as it can be something that you want to improve. Of course, you want to earn more orders, higher average order value, that’s kind of those kinds of experiments, then a success will be actually increasing, increasing those numbers. But there are often also experiments where you want to implement something. And basically you run an experiment more or more as an insurance to make sure that nothing goes wrong. For example, when you when you would do a redesign, if for whatever reason, you want to do a redesign the media ideally, of course, those numbers would go up with that in mind, historically speaking, redesigns don’t necessarily are the best way to to increase your numbers, but at least you want to make sure it’s not going to be worse than what you already have. And that’s something you could run an experiment on. So you don’t have a single thing changing, a lot of things are changing. But I want to make sure that nothing goes downhill. And that’s something you can also run experiments on. So the goal of the experiment, don’t necessarily to improve something, but there’s to make sure actually, that everything stays the same, as much as as possible. And then that will be a success. So depends a bit on the on the kind of experiment that’s that you’re running.