Mats
Why are some websites performing bad on mobile devices?
Recently Google released a very interesting new performance metric called INP (Interaction to Next Paint). INP puts focus on responsiveness and logs the latency of all interactions throughout the entire page lifecycle. It simply checks how fast a site responds to user inputs and gives the user visual feedback, something which is crucial to the user experience.
90 % of a user’s time on a web page is spent after it has loaded, which makes this metric a key value for all web site owners if you want to deliver a good overall user experience.
What is a good INP Value?
First, you need to know that INP aims to calculate an overall interaction latency by selecting one of the single longest interactions when a user visits a page. As in every other Core Web Vitals, there are limit values that ranks as good performance, performance in need of improvement and as a bad performance.
INP below 200 milliseconds ranks as a good performance
INP above 200 millisecond milliseconds but below or at 500 milliseconds means that your page's responsiveness needs improvement.
An INP above 500 milliseconds means that your page has poor responsiveness.
So, now when we know what INP is and what a good score is, it’s time to check how different technology and CMS affects INP on different devices (desktop and mobile).
Before we begin, I want to clarify that we benchmarking the actual website built with a CMS, not the CMS editor view itself.
INP on a desktop
Let’s start by comparing different CMS on desktop by using Core Web Vitals technology report.
First, let’s look if we can find any difference between a traditional CMS compared to a headless CMS on desktop. Below you see some of the most common CMS brands in the world and some local Swedish brands as well. As you can see, they all perform well, and you notice no big difference between them. If you are a bit harsh, you might say that Optimizely has some issues to work on.
Let´s do the same check with some of the most common headless CMS and compare it with the best traditional CMS in the first test (Sitevision CMS). As you can see, we get a slightly poorer performance from the headless CMS, but no one is dropping too much to call it a bad performance. Our best guess on this small drop in performance is that more work is being done on the client side when it comes to rendering and collecting data through APIs. And a wild guess is that websites built on headless technology, more often uses different modern JavaScript frameworks like React and Vue.
Even though we see some small performance differences between traditional CMS and headless CMS, most of all benchmarked CMS are doing well when it comes to the INP metric. Or you might say that the people who are creating the websites are doing a good job.
INP on mobile devices
Let´s do the same comparation for mobile devices and check if we can find any big changes in performance.
As you can see below, we get a completely different graph comparing to desktop and we can confirm that Wix has a big challenge when it comes to INP on mobile devices. And we see that WordPress, who is the most common CMS in the world, drops from almost 100 % to 60 % on mobile.
So, what´s happening with headless CMS when we shift to mobile devices? As in previous comparation we added the best performing traditional CMS on mobile, which in this case is the same as we added on desktop (Sitevision CMS). As we can see in the graph below, we have a big drop in performance when it comes to INP in headless CMS. In this case Contentstack has the biggest challenges but no one is reaches above 50 % in this example.
So why are we seeing such a bad “user experience” on mobile devices?
Let´s start by checking if we can find out what´s slowing down the mobile versions. As we mentioned earlier, we guessed that headless CMS are using more modern JavaScript Frameworks like React and Vue for enriching the user experience, so let’s check if we can see any parables by investigating how different JavaScript Frameworks affect INP on desktop and mobile.
We can clearly see the same pattern when we compare different frameworks on desktop and mobile in the graphs below. By looking at the different graphs, we believe it´s not a wild guess to assume that this kind of frameworks has an impact on INP performance on mobile, but still, it´s just a guess. Another thing we noted during the comparation between different frameworks was that you cannot simply do a benchmark like this and chose a framework based on the numbers below and hope to get the best INP score. It´s much more complex. You need to consider what framework that suits you best when it comes to purpose and knowledge. To exemplify, we added lit-elements which are known to be one of the fastest frameworks on the market and as you can see below it performs terribly bad.
The INP metric has only been available in Core Web Vitals technology report for a couple of month and site owners are seeking answers to this behavior on mobile devices. Google got this question from a user and their answer was quite clear and, in some way, obvious.
“Desktop devices tend to be orders of magnitude more powerful than mobile devices, and so they can process code (both app code and browser code) much faster, which leads to faster response times. Mobile devices tend to be much slower, and so the same page running the same code will usually report longer response times for the same types of interactions.”
Another reason for mobile devices performing bad, is of course slow connections in combination with large amount of data transfer, so be sure minimizing the amount of data by using modern image formats, remove unused JavaScript and minify the scripts you are using. Then, if possible, put your scripts in the bottom of the page to prevent scripts from blocking calls which may delay the page rendering.
But when it comes to why different framework differ in performance it´s not that obvious. You need to dig deeper into each website to find explanations but maybe one reason could be the ripeness of the framework and community knowhow that comes in to play. In other words, if you are skilled enough in any framework you will probably be able to build a fast site, at least when the user access it from a desktop.
Check your INP metric
The easiest way to check your websites INP score, is by installing the Web Vitals Chrome plugin on your desktop. I recommend activating the option “Comparing local experiences to phone field data”. This option helps you get a pretty good feel for how many of your mobile users getting a good user experience, and how many of them getting a crappy one.
In the screenshot below you see how the report looks like when you activate this option on your desktop in Chrome.
If you want to get into more details about framework performance, then this public benchmark page is a good start for you. As you can see on this page, Vanilla is the right path to follow if you are searching for speed!
Good luck with improving your INP performance!