What is lab data?
Lab data is the name that the performance community uses for measuring performance in a controlled and synthetic environment. Or you know, a lab😉
It can mimic users' conditions - to a certain extent - and help you gain insights into what could happen in regards to web performance of your site if a user would visit it in its set conditions. It can be great to debug performance issues, as you can do a replay frame-by-frame, or signal issues that might arise in your code before the site goes live.
It is however not your real user, it's simulated and it can't account for a lot of factors a website visitors might experience, like interactivity, third party influences or conditions that might arise after a page has been loaded. For this you can utilize data gathered from your real users. A famous example of a lab data tooling is Google's Lighthouse, but there are other solutions like GTmetrix and Webpagetest that rely on lab testing. We'll look into their features, benefits and shortcomings later.
What is field data?
Field data is what’s actually used by Google’s Core Web Vitals. It measures the experiences of real website visitors - your real users. There are a lot of factors that cannot be measured in a controlled environment in a lab data test. That's why Google introduced the web-vitals and its metrics.
The great thing about field data, is that it captures the experience of all your real-world visitors who naturally all have different devices, network conditions and geographic locations. All these factors, and many more, can impact their experience and thus how your website's speed is perceived by your users. By taking into account fast as well as slow user experiences, field data provides a complete image of performance data from your users. Real user monitoring solutions, like RUMvision, also rely on field data to get you insights. They are in fact the solution to deepdive. But first, let’s discuss the Chrome User Experience Report (CrUX).
Because field data is based on real-user visits, it reflects the actual devices, network conditions, and geographic locations of your users. Field data is also commonly known as Real User Monitoring (RUM) data; the two terms are interchangeable.
Visual explainer
When to use lab or field data
Knowing when to deploy the right tool is valuable. That way, it can save you a lot of time and headaches. My own rule is to always use lab data supplementary to RUM data when a website is live.
Example of how to use RUM and lab data together
Suppose a website has a high Interaction to Next Paint, I can see this in CrUX data. However, I then have no idea why my INP is on the high side. This is what I use RUM data as well as attribution and, of course, the LoAF API for.
Now I know which elements people actually interact with and thus produce high latency. That way, you don't have to guess where a high INP comes from Now that I have this data, I can use DevTools to create a performance trace and see what exactly is happening.
Also read: Using your RUM LoAF data for INP debugging purposes
Staging environment
When a website is under development, the value of RUM data sinks because there are only developers working on your website. These often work on desktop under fast internet connection, for example wired internet in the office. On a staging environment, it is useful to stick to best practices, for example those Google Lighthouse shows you. In addition, it is never a bad idea to hire a consultant to do a quick scan on your staging environment to pick out UX deteriorating factors before going live.
The minute your website goes live, it is useful to deploy a RUM tool. This is because in the beginning, there are still many issues that come to light and need to be fixed. When you depend on CrUX data here, you have to wait until there is enough data and you are behind the times because of aggregated data.
Search ranking (SEO) purpose
When your approach is to engage with the Core Web Vitals to improve SEO, it is good to know that you are always and only looking at field/ RUM data, for example through Pagespeed Insights or Search Console.
An 85 score or even a 100 score on Lighthouse is not going to contribute to better rankings. To get SEO benefits from Core Web Vitals you need to pass:
Different Core Web Vitals tools
There are lots of different types of tools to measure your Core Web Vitals with. The important thing to grasp is that each tool can use three types of data.
- Lab data (synthethic)
- CrUX data (real users, but delayed and not detailed)
- RUM data (real users, real-time, and able to deep-dive)
RUMvision vs. CrUX
The low cost way to collect field data is by using Google’s free data: Chrome User Experience Report (CrUX). It's insights are limited, but mighty and defintely the best starting point when you have enough traffic on your website. CrUX data is also the data that is used for passing or failing the Core Web Vitals assessment.
With CrUX data, you can get data from mobile and desktop traffic from the entire origin, homepage or the exact URL you enter. The nuance is that the exact URL must have enough data to be in the CrUX data set. For example, the blog overview page on web.dev has enough data. If you have a webshop with a lot of products then all the products together will have enough data, but individual product pages will not. Thus, it becomes difficult to see data from specific URLs with CrUX. There's no clear number of how many traffic a specific URL.
Advantages:
- It shows you where your real users experienced performance or UX issues, as it's based on field data.
- It shows you users experience over 6 key performance metrics (soon to be 5 when FID gets removed), that will help you discover where you should optimize.
- You find the 3 Core Web Vitals that are used for SEO ranking (LCP, INP, and CLS).
Core Web Vitals tool for historical data
Disadvantages:
- As the name might suggest, CrUX only measures user experiences on browsers that run on Chromium. Experiences on Safari or Firefox are not part of this dataset.
- In fact, it only reports back after you've had enough data for Chrome visitors that month.
- The Core Web Vitals are shown aggregated over a 28 day period. That means that a code change or third parties impact only becomes fully visible after 28 days.
- It only shows the experience that people had on the 75th percentile. This means you’re missing insights into 25% percent of your visitors' experiences.
Although CrUX-based tools are a good starting point for investigating Web Vitals performance, we strongly recommend supplementing it with your own RUM.
RUMvision vs. Google Lighthouse
The biggest difference between Google Lighthouse and RUMvision is the fact that Lighthouse operates with lab data and RUMvision with field data. Therefore it measures 2 completely different things. We wrote a blog about "When to not use Google Lighthouse" to get a better understanding of its use-cases.
Often Lighthouse’s data comes from a device with slower internet than your own users, as such, it might not accurately represent the experience of your users.
Another difference is that Lighthouse does not take Time to First Byte (TTFB) into account. TTFB measures the time it takes between the first request for a source and when the first byte of that source begins to load. RUMvision does measure TTFB because it is one of the most noticeable factors for your users when loading takes a long time. These are the metrics that Google Lighthouse captures:
During a Lighthouse test the “user” does not interact with the page, this could introduce blind spots for metrics like the Cumulative Layout Shift (CLS) and Interaction to Next Paint (INP) that can happen at any given time. It only measures the first 5 seconds of a pageload. The closest Lighthouse has to offer for the INP is a Total Blocking Time, but the results will also differ from what real users experience.
If you want an idea of which script was actually harming during user interaction, with a button in example, the easiest way is to install RUM.
Apple.com score on Google Lighthouse
RUMvision vs.GTmetrix
Even though GTmetrix technically uses lab data they also try to simulate a real user's experience and behavior on a site. You can change these “user’s” variables like location, connection speed, etc. By doing this GTmetrix gives you the option to select variables of what you think represents your own audience. Keep in mind that this could introduce blindspots, as you won’t know which type of visitors experience bad core web vitals on your own website.
The GTmetrix score can also differ from other lab data tools like Google Lighthouse and WebPageTest, because all of them have a different methodology.
With RUMvision you can see which type of visitors you have on your website and how this correlates with web performance metrics. Not every user will be on a fast or slow device, but you have inbetweens as well. Therefore it is useful to capture visitors by the fact that they entered your website via an ad, are they first time visitors, what their internet speed is, and how fast their device is.
