CRO’s Evolution from Silo to Business Strategy and Innovation: Insights from Ruben de Boer

At VWO, we sincerely believe in building a powerful community for broad learning and sharing. Our ‘CRO Perspectives: Insights from Industry Thought-leaders’ blog series brings together some of the brightest minds in the field of conversion rate optimization to share their stories, expertise, and best practices. Our goal is to help you derive effective, essential learning through their answers. We hope this post will be a treasure trove for you to apply these insights in your work. 

Leader: Ruben de Boer

Role: Lead Conversion Manager at Online Dialogue 

Location: Amsterdam, Netherlands

Speaks about:  Conversion optimization • Digital strategy • Experimentation • Business growth • Change management 

Why should you read this interview?

Ruben is an accomplished Lead CRO Manager and consultant with over 15 years of experience in data and optimization. At Online Dialogue, Ruben leads the Conversion Managers team and manages conversions for renowned clients like Vodafone, eBay, and Rabobank. Through his company, Conversion Ideas, he provides high-quality online courses to help beginners kick-start their career in CRO. 

As a bestselling Udemy instructor and sought-after speaker, he shares insights on experimentation culture, change management, and conversion rate optimization. Ruben’s passion lies in connecting with like-minded professionals globally and empowering others to use CRO knowledge for a positive impact.

This interview is packed with insights from Ruben’s extensive experience and expertise, offering valuable perspectives on conversion rate optimization. Whether you’re new to the field or a seasoned professional, Ruben’s interview promises to enrich your understanding and inspire new strategies for success.

Learning on the journey of CRO

I have been in the business of optimization and experimentation for over 15 years. So everything has changed. Going from all the mistakes one can make when starting with CRO to now seeing it as something much broader than just a trick or tactic to increase conversion rates. 

I now see experimentation at many levels, not just marketing. Experimentation helps to make better decisions and build better products. 

Clients have moved experimentation from marketing to product. It helps to make better decisions and build better products. 

Meta-analyses in experimentation

At Online Dialogue, we A/B test to earn and learn.

We want to have a positive ROI (which we always have), but we also want to better understand our client’s customers to develop better hypotheses and align better with the business. 

At Online Dialogue, we work with multi-disciplinary teams. We have Managers, Analysts, Psychologists, UX experts, and Developers. Everyone is a consultant, and everyone contributes to the success of A/B testing and learning. Due to this unique combination of specialists and our process, we are able to gain true insights and learn fast. 

This is strengthened by excellent documentation in Airtable. Combining the data and results of multiple A/B tests (meta-analyses) helps us learn what works where in the customer journey and how. Such insights benefit many stakeholders in the organization, such as online marketing, offline marketing, sales departments, and brick-and-mortar stores. 

If you are not using Airtable yet, you can check out our free course on this topic.

Increasing commitment to A/B testing

Looking solely at the Netherlands and award-winning cases (like the Experimentation Culture Awards), A/B testing is being taken more seriously and growing. 

To illustrate this: 

5+ years ago, we received many questions from the Marketing and E-commerce departments to conduct their entire CRO program. 3-4 years ago, companies started to understand it is not a one-person job, so they hired some specialists and asked us to complete the team with our specialists and set up a process. 

Today, we receive more and more questions from product teams to help them experiment. We enter the company at a higher level, operate in the Centers of Excellence, set up teams and processes, and work on the scope, strategy, and culture. 

I hope more and more countries will start following this trend. 

Digital accessibility in CRO

In several databases (including Online Dialogue’s with over 10,000 A/B tests), I see that experiments related to Usability have a very high win rate. Usability principles and accessibility principles are very much alike. So if you don’t take accessibility into account from a moral perspective (which you should), then do it to increase conversion rates.

Digital accessibility is a moral obligation and, in an increasing number of countries, is now also a law. Many people with impairments are unable to use many websites. It is our job to help them. 

Defining success in CRO

In the beginning, it can be very beneficial to run many small experiments. This helps the team learn the process. Next, many organizations start with an ROI goal, which is also fine if higher management demands it. 

Once an organization takes CRO more seriously and invests more resources, the number of experiments and ROI can no longer be a success metric. When onboarding more teams, time is needed to educate them, and as they have less experience, the quality will decline. A better metric could be the number of teams testing. 

Eventually, the cost of experimentation or the percentage of changes in the organization validated could become a good success metric. 

Tips for effective experimentation

A common mistake is to skip it, or not do it enough, and rely on best practices. 

The more insights you get from your research, the better your hypotheses resulting in more winning A/B tests and greater learnings from those tests. 

Also, conduct scientific research. There is so much research about your market, product, and target audience publicly available, but it is barely used in CRO. Especially with help from AI (like Consensus, Scite, and Elicit), there is no excuse not to do it. 

Experience using VWO 

It is a great tool. The user interface is very nice, and it contains many features. It’s great that heatmaps, recordings, surveys, and form analyses can be part of the package. 

The future of CRO

In the next few years, hopefully, more markets will start with CRO or take it more seriously. For more mature markets, there continues to be a higher demand for moving experimentation from its silo into product teams.

CRO specialists will need to let go of their strict process and embed it in the process of the product team. For companies that are already there, they might start moving experimentation into strategic decision-making and innovation. 

Looking at technology, AI will likely make everything more efficient. We have to wait and see how far this will go. I hope that AI can help us draw better learnings from many experiments combined so we make better products for happier and healthier lives. Also, personalization (like recommenders) might improve due to AI. 

In the very far future, who knows what optimization will look like? Perhaps websites will not be as important as it will just be bots talking to bots.