Jeff from WP Tavern has been bugging about the results ever since I published my original post, so I guess this will be a lucky day for him 🙂 I actually signed up to do this talk to give myself a hard deadline to finish this (or, at the very least, make some progress).
As with every pet project, Measure Jetpack progressed to all the usual steps – initial entusiasm, lack of time to do it properly, neglect and a loss of ambition. So the results I currently have are nowhere near I initially envisioned them to be in scope and quality. But at least there are results.
I’ve setup a testing environment. I took a completey ordinary Digital Ocean droplet (1GB RAM, 1 CPU, 30GB SSD) that I was not using anymore. It also had a pretty ordinary LEMP stack (Ubuntu 14.04.4 LTS, nginx 1.4.6, PHP 5.5.9, MariaDB 5.5.47).
I’ve installed the latest (4.6.1) version of WordPress, and the latest (4.3.2) version of Jetpack and activated Twenty Sixteen 1.2 theme. Just so the site does not stand completely empty, I’ve created 600+ posts by borrowing content from Wikipedia – descriptions of all world countries, capitals an some major cities.
I built a small plugin to do the actual measurements, that collects all the data and stores it in a CSV file. I’ll be sharing it on Github a bit later, so anyone will be able to replicate my results if they ever wanted to.
For every module, 9 pageviews were generated. The lowest and the highest values were discarded to eliminate possible fluke results. And an average was calculated from the remaining 7 runs.
Just installing and activating Jetpack will add 0.47 Mb of peak memory usage. Compared to 3.33 Mb usage of a plain WordPress installation that is a significant increase. Which gets even more significant if you turn on the Recommended modules (2.04 Mb) or go all-in and activate every single module included with Jetpack (2.451 Mb).
There also is an increase in PHP execution time. On average, Jetpack adds ~0.076s to it. It seems very small in absolute terms, but relative to results plain WordPress on the same system (0.18s), it would be a significant change, too,. But from scientific point, there is a big variance of theese results (+- 30%), so it falls within the margin of error.
Going in, I was expecting Jetpack to be adding quite a few database calls, But as it turns out, even turning every single Jetpack module only adds 10 SQL read queries to the blog homepage. Again, it might seem a lot, compared to a plain WordPress, but in nature, WordPress home page can have anwyere from several dozen to 200+ SQL read queries, so it is not really that significant.
Activating all Jetpack modules will add 13 (11 internal, 2 external) scripts and 11 internal styles and that is quite a lot. But this is being done by only a handful of modules: Infinite Scroll, Gravatars, Photon, Likes).
The last number I was measuring was remote HTTP requests. As some parts of Jetpack are based on external API’s, I was expecting quite a few of those. But at least on the home page, I did not find any remote requests happening.
- Jetpack will add at least 0.5 Mb (and up to 2.5 Mb) of peak memory usage on home page of a WordPress site;
- There is an effect on execution times, too, but a bigger sample size is needed for reliable comparison;
- Only a handful of Jetpack modules load additional assets to WordPress home page;
- Jetpack adds no remote HTTP requests on WordPress home page;
As you have probably noticed, I haven’t tested any alternative plugins for comparison. I intend to do just that in the next step and this is where you can help. I could go the easy way and compare Jetpack to the most popular stand-alone plugins in respective categories, for example Contact Form 7 for forms, Disqus for comments, etc. And I am going to do just that. But I am pretty sure Jetpack will come up ahead in those comparisons just because the stand-alone plugins have way more features and options in them.
But I also want to find the most similar stand-alone plugins for every Jetpack to make a more fair test. So if you know of plugins that are very close to Jetpack modules, please drop me a line at firstname.lastname@example.org. That would really be a big help, as I do not have the time to go through all 40k of plugins myself.
Another thing that became clear to me during this phase, is that measuring only the homepage is not nearly enough. A lot of the modules do their dirty work in other places of the site, like single page or the admin dashboard. So I also intend to get some numbers on those places, too.
The question I always get asked whenever I talk about Jetpack is wether it is a good thing or a bad thing. Or if we should use it or not.
Personally I do not like to use it much, but that does not mean that Jetpack is bad. Everything depends on your goals.
If you want control and flexibility, Jetpack is probably not for you. It’s modules have very little in terms of configuration, and there is not so much you can tinker with.
But if all you care about is stability and reliablity, Jetpack offers features that just work and will stay working after the next release. And the next. And the next. It is the kind of plugin that you can install and forget.
Below you can find my slides from the Stockholm talk and the video of it.