The impact of Google Optimize (and any other client-side A/B testing) on Core Web Vitals, is a topic of interest for many website owners and developers. Especially when combining client side A/B testing with so-called anti-flicker snippets or async-hiding snippets.
Introduction to how client side A/B testing works
Anti-flicker snippets or async-hiding snippets will hide the page while loading and deploying A/B tests. Once done deploying or when reaching a time-out, the visited page finally becomes visible.
The theoretical impact
It might then be easy to understand how this could impact the duration of a white screen. This is known as First Contentful Paint (FCP). When FCP is pushed back for whatever reason, the following is impacted as well:
- the amount of time users are looking at a white screen;
- perceived performance and early user engagement;
- and bounce percentage as a result.
The information presented in this blog post stems from an analysis of mobile data to determine how A/B testing with Google Optimize affects FCP and other relevant metrics. Such as Largest Contentful Paint (LCP) and Time to First Byte (TTFB). The results indicate a significant improvement in FCP, with a positive impact on LCP as well.
+30% FCP gain - an A/B test use case
A/B testing and pagespeed metrics
While there are many pagespeed metrics, A/B testing would typically only impact a subset of the metrics. These are as following:
Time to First Byte
A/B testing does not influence TTFB. That's because the A/B testing process begins only after the bytes was received that is containing A/B testing instructions (hiding the page while loading and deploying A/B tests) for the browser.
In other words: TTFB already happend before, so there is no relation between client side A/B testing's async hiding snippet and TTFB.
First Contentful Paint
The FCP metric is the most affected by A/B testing, as anti-flicker snippets sits in the <head> and deliberately hides the screen. The browser might be able to render the page already using such snippets. However, the page will visually stay hidden for a while.
Largest Contentful Paint
LCP is about the biggest (block level or image) element within the user's viewport, before user interaction takes place. You would want such contents to appear as soon as possible.
A so-called async-hiding snippet could help to mitigate LCP delays. Nevertheless, because of the amount of work typically going on in browsers, LCP are often negatively impacted as well when using client side A/B tests.
How A/B testing is impacting your Core Web Vitals
Anti-flicker snippets are small pieces of code used to minimize the flickering effect that sometimes occurs when running A/B tests or website experiments. Flickering is the brief display of the original content before the variations are loaded, which can affect user experience and experiment results.
The anti-flicker snippet typically hides the page content for a short period to give the experiment script time to load and apply the necessary changes.
Here's an example of the Google Optimize anti-flicker snippet:
<style>.async-hide { opacity: 0 !important; }</style>
<script>
(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;
h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
(a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);
})(window,document.documentElement,'async-hide','dataLayer',4000,
{'GTM-XXXXXX':true});
</script>
The number 4000 indicates the maximum hiding time in milliseconds (4 seconds). You can adjust this value to suit your needs, but keep in mind that higher values can negatively impact user experience.
Google preferences, penalties, and competition
First of all: We don’t see why or how Google would understand the need for A/B testing. Even when done for a short amount of time (as the Google docs are recommending as well), it’s still likely to impact UX during that time.
And we, as an industry, do know Google understands the need for optimal technical UX / web performance. They announced and introduced this publicly and is called Core Web Vitals.
Your site not being penalized when running A/B tests could be correct. However, one also won't get a ranking boost when not meeting Core Web Vitals thresholds. Which is where competitors might then outperform them.
Also because next to sitespeed and performance, regressed UX would then also impact:
- conversion (which could be sign-ups or revenue);
- customer retention;
- SEO as well as SEA/CPC.
So, let’s shed a light on the A/B testing versus performance subject.
Conclusion
While client-side A/B testing with anti-flicker or async-hiding snippets can provide valuable insights into website performance and user behavior, it is essential to understand its potential impact on Core Web Vitals. By carefully considering the implementation of A/B testing strategies, website owners and developers can minimize negative effects on key metrics and user experience. Ultimately, balancing the need for experimentation with the importance of maintaining optimal website performance is crucial for fostering better user engagement and achieving online success.
