Many agencies struggle with performance conversations because they rely on synthetic testing alone. Here's how Real User Monitoring changes the game and what we've learned communicating these insights to clients.
This post is written by our Gold Partner run_as_root
David Lambauer is the founder of run_as_root, a remote-first agency specializing in high-performance Magento 2 and Hyvä storefronts. They use RUMvision to monitor real-user performance across client projects and bridge the gap between technical optimization and actual business impact.
Lighthouse scores vs. real user experience
Every agency knows this situation: a client asks about site speed, and you are left wondering whether they want to invest in actual fixes or just need a green score on a dashboard they do not fully understand. The tricky part is that many agencies still rely purely on Lighthouse scores - metrics that often fail to show what actual visitors experience on the site.
After three years of building Magento 2 and Hyvä shops at run_as_root, one thing has become clear: discussing performance with clients is not about showing abstract numbers. It is about understanding what real visitors run into and adjusting how you communicate with each client. Real User Monitoring makes that conversation a lot simpler.
How we talk about performance with different clients
1. Clients who just want the business impact
These clients trust your agency to handle the technical side. They want quarterly updates, high-level context, and practical recommendations. Technical details matter less to them than commercial results.
How we handle communication:
- Focus on commercial impact: conversion changes, revenue trends, and user experience trends
- Keep updates short with clear summaries and business cases
- Provide concrete recommendations for the upcoming quarter
- Leave out unnecessary technical jargon
Example update:
"The Core Web Vitals optimizations contributed to a 12% lift in mobile conversions last quarter. For next quarter, we recommend investing €8K in image workflow improvements to capture an estimated €45K in additional revenue."
2. Clients who follow every deployment
These clients want to know how each release impacts performance. They follow deployments closely, ask practical technical questions, and want to see immediate data.
How we handle communication:
- Share before-and-after comparisons for relevant releases
- Break down Core Web Vitals per page type and template
- Explain what code changes were made and why
- Use automated CI checks alongside RUM to track changes over time
Example update:
"After updating the product image loading strategy yesterday, mobile Largest Contentful Paint improved by 1.2 seconds in lab tests. More importantly, RUMvision shows that 23% more real mobile visitors now pass the threshold. Here is the breakdown per template and what we are addressing next."
3. Clients who do not ask about performance at all
These clients do not actively ask about performance. Even so, your agency still needs to protect site quality and your own delivery standards.
How we handle communication:
- Keep performance status as a natural part of general project reporting
- Position performance work as standard code quality and maintenance
- Connect speed improvements to goals they do care about, like SEO visibility and ad bounce rates
- Factor baseline performance work directly into project budgets
Example update:
"During scheduled maintenance, we restructured image delivery. While the synthetic score moved up, the practical result in RUMvision is that actual shoppers load the main product visual 34% faster. That directly benefits campaign bounce rates and search visibility."
Handling budgets and scoping
Most merchants do not maintain a separate line item labeled "performance optimization." Speed work frequently gets squeezed into maintenance budgets or postponed as technical debt.
In our experience, agencies handle this successfully in three ways:
- Building performance standards into every project scope from day one (our default approach)
- Tying optimizations directly to conversion and revenue data
- Using verified client speed benchmarks during sales pitches
At run_as_root, we treat performance as a foundational standard rather than an optional add-on. Every build follows the same baseline setup, includes performance testing, and ships with clear benchmarks. We do not ask clients whether they want a fast site - we build it into the way we deliver projects.
Our default Magento 2 performance stack
In day-to-day agency work, roughly 80% of Magento 2 performance bottlenecks come down to choosing the right infrastructure stack and configuring it properly.
The remaining 20% involves deep frontend work, template tuning, and custom caching logic. Getting that first 80% right gives your team a clean baseline to build on.
Backend tooling for TTFB
We rely on a standardized setup that reliably handles Time to First Byte (TTFB) across our Magento 2 builds:
1. Redis for sessions and cache
- Replaces MySQL session storage for faster concurrent handling
- Provides low-latency cache retrieval (typically sub-50ms)
- Typical outcome: Server response times reduce from 800-1200ms down to 200-400ms
2. Varnish with custom templates
- We use Peter-Jaap Blaakmeer's Varnish configuration (developed alongside Thijs Feryn from Varnish)
- Serves cached catalog pages directly without executing PHP or querying the database
- Maintains full-page cache hit ratios above 90%
- Typical outcome: Cached responses return in 50-150ms
3. Cloudflare CDN with Polish
- Low base entry cost on Pro plans
- Automates WebP delivery and asset compression
- Caches static assets closer to shoppers, cutting network latency
- Polish reduces image transfer weight by roughly 60-70%
4. Tideways for profiling
- Profiles a targeted sample of production requests
- Flags slow SQL queries, heavy third-party modules, and memory issues
- Sends alerts when execution times exceed thresholds
- Catches backend regressions before they impact the user base
5. RUMvision for real-user monitoring
- Monitors actual user experience across all real devices and networks
- Tracks Core Web Vitals directly from active shoppers rather than simulated lab runs
- Uncovers issues limited to specific user segments, URLs, or campaigns
- Highlights the difference between lab assumptions and live field conditions
- Monitors third-party script overhead and compares speeds against competitors
Frontend setup with Hyvä
Once backend delivery is fast and cached properly, Hyvä handles the frontend overhead that traditional Magento themes struggle with:
- Lean scripts: Drastically smaller JavaScript payloads compared to Luma
- Core Web Vitals: Built from the ground up to pass Google thresholds cleanly
- Better mobile responsiveness: Significantly improved interaction and render speeds on real mobile devices
Having a standard stack in place saves dozens of hours of repetitive debugging on every new client shop.
Where synthetic audits fall short
Lighthouse tests are useful benchmarks, but they only test one clean scenario: a single device emulation under a simulated network profile. Real visitors browse on mid-range phones, unstable 4G connections, and during busy flash sales. Relying only on synthetic audits leaves blind spots that can hurt client trust.
There are numerous tools available for measuring the sp…
We saw this directly with a merchant whose synthetic audits scored a clean 95. In RUMvision, however, 40% of their actual mobile visitors were failing Core Web Vitals. The lab test simply could not capture the mix of devices and network conditions in their real customer base.
Case study: campaign tracking broke our client cache
This is where the remaining 20% of performance optimization comes in, and where field data is critical. On one client site, our synthetic checks showed clean metrics across the board: fast TTFB, low server response times, and green Lighthouse scores.
RUMvision, however, showed a clear discrepancy: visitors arriving via paid marketing campaigns (containing tracking parameters like ?utm_source=google&utm_campaign=spring) experienced far slower load times than direct or organic visitors. Because of the query parameters, requests bypassed Varnish entirely and hit PHP directly on every click.
Our data shows that the question should actually be: by…
Standard synthetic tests missed this completely because automated audits did not run against paid landing URLs. Without real-user data, the shop would have continued paying for expensive ad clicks while serving cold, uncached responses to new shoppers.
The solution: We updated Varnish to strip marketing parameters before evaluating cache keys. The outcome: Load performance for ad traffic improved by 78%, and conversion rates on paid campaigns lifted by 23%.
Proper tooling gets a site 80% of the way there. Identifying the remaining edge cases - the ones that directly affect revenue - requires real-user data.
Using RUMvision gives agencies clarity on:
- Performance variance between devices and connection qualities
- Geographic latency differences missed by local testing
- Response degradation during traffic spikes
- Which specific user segments experience slow interactions
- The commercial impact of speed improvements
The takeaway: It shifts client discussions from "the tool gave us a green score" to "here is how your actual shoppers experience the store, and what that means for your bottom line."
