Measuring Your WordPress Cache Hit Rate

Caching is one of the key ingredients for great WordPress performance, but how do you find out whether your cache configuration is efficient? In this tutorial we’ll explore some of the tools and methods which you can use to measure your WordPress cache hit rate.

WordPress Cache Performance Graphs
WordPress Cache Performance Graphs

Adding Page Caching Headers

There are many great plugins and solutions out there for full page caching in WordPress, and it’s not easy to cover all of them in this post. But most of them have one thing in common — they all serve requests (both cache hits and cache misses) that originate from the web server software, and most web servers will write things to the access log.

If we can somehow, upon serving a request, tell the web server whether it was a cache hit or a cache miss, we can pass that information and have it eventually end up in our log files. A custom header is the perfect solution for this:

header( 'X-Cache-Status: miss' ); // cache miss, by default
header( 'X-Cache-Status: hit' ); // overwrites the previous header with a hit

The cache miss header can go pretty much anywhere in your application, before anything is output on screen (or written to the output buffer). If you’re unsure, the wp-config.php file is a good place to put it, right before loading the main wp-settings.php file. That way the header will be output before advanced-cache.php (the WordPress page caching dropin) is loaded.

The cache hit header is more tricky. You’ll need to put this in a place where you know for sure, that the content is being served from cache, and this will vary between different caching plugins. A good place to start looking is around the call to the header() PHP function.

We’ll try to address the cache hit header for various popular caching plugins below, and after that we’ll explain how to get these headers written into your access logs, and how to analyze those logs later on.

Jump to:

Note that some plugins will have a “debug” mode available, which we advise against in production environments, since it may cause unwanted overhead, unlike simply adding a single header.

W3 Total Cache

W3 Total Cache is one of the most popular caching plugins out there. We’re not particularly fond of its architecture, upsell nags, and the billion configuration options, but it does work fairly well for a lot of people.

At the time of writing, the latest version of W3 Total Cache uses a class called W3_PgCache to handle page requests at the advanced-cache.php level, so that’s where we’re going to look. There’s a method called process_cached_page() which is most likely the one responsible for serving a cached request, and our “cache hit” header can go right after W3TC is done outputting its own cached headers:

/**
 * Send headers
 */
$this->_send_headers($is_404, $time, $etag, $compression, $headers);
header( 'X-Cache-Status: hit' ); // It's a hit!
if (isset($_SERVER['REQUEST_METHOD']) && $_SERVER['REQUEST_METHOD'] == 'HEAD')
    return;

To check whether it works or not, try requesting a page with cURL from the command line a couple times, and watch the response headers:

$ curl -v http://example.org/ -o /dev/null
< X-Cache-Status: miss

$ curl -v http://example.org/ -o /dev/null
< X-Cache-Status: hit

Other page caching methods in W3TC will likely work as well, but we have not tested them.

WP Super Cache

Another popular caching plugin is WP Super Cache by Automattic. It’s more lightweight than W3TC but still has a fair amount of options, modes and whatnot. The codebase is much smaller though, so it’s easier to find a good place to plug our cache hit header, in wp-cache-phase1.php right near its own fancy header:

header( "WP-Super-Cache: Served supercache file from PHP" );
header( 'X-Cache-Status: hit' ); // It's a hit!

Again, we found this simply by looking through the codebase for calls to header().

If you’re working with Apache and WP Super Cache’s mod_rewrite mode, you’ll have to have the cache hit header somewhere in your .htaccess file instead, because cache hits with the mod_rewrite mode are files served directly by Apache, without hitting PHP at all. In this case your best bet is to serve a cache hit header from .htaccess with a Header directive, and have your miss header() call overwrite that in your wp-config.php.

WP Rocket

WP Rocket, the new kid in town, also has full page caching capabilities via WordPress’ advanced-cache.php dropin. Finding the right spot for the cache hit header was also straightforward by looking for other header() calls. This well-named rocket_serve_cache_file() function is located in wp-rocket/inc/front/process.php:

// Check if cache file exist
if ( file_exists( $rocket_cache_filepath ) && is_readable( $rocket_cache_filepath ) ) {
    header( 'X-Cache-Status: hit' ); // It's a hit!
    // ...

Redis Page Cache

Our very own Redis Page Cache plugin is a less popular full page caching option, which is extremely fast and uses a Redis database to store cached requests. Luckily, the plugin itself already outputs cache hit/miss statuses using the X-Pj-Cache-Status header, so no changes required here.

Nginx proxy_cache and fastcgi_cache

If you’re using Nginx’s proxy_cache or fastcgi_cache for full-page caching in WordPress, you already have the necessary $upstream_cache_status variable which you’ll be able to use in the log format, but if you’d still like to output a status header for debugging purposes, you can:

add_header X-Cache-Status $upstream_cache_status;

If you use Nginx’s fastcgi_cache or proxy_cache in addition to a WordPress-level caching plugin (like we do here at Pressjitsu), you might want to use a different header for the two levels, so that you can measure the efficiency of each.

Others

There are many other caching plugins and solutions available for WordPress, and unfortunately we can’t cover them all. But the above examples should give you a pretty good idea of where the cache hit and miss headers should go.

Please note that in some of our examples above we’re modifying existing plugin code, which is against WordPress best practices, since plugin updates will overwrite any of your changes. Since most plugins tend to serve cache at the advanced-cache.php level which runs very very early, unfortunately there’s really no easy way to hook into a cache hit from outside of the caching plugin itself.

A more complicated but elegant way of achieving similar results would be to reverse the headers, i.e. serve a cache hit header by default from wp-config.php (and hope that the caching plugin doesn’t blindly strip all your headers) and then change it to a cache miss during the WordPress init action, though this may likely not be very accurate.

Another alternative approach would be to hook a shutdown function (as early as wp-config.php) and then inspect the contents of some global/static variable (or even a debug backtrace) and figure out whether it was a cache hit or miss. At this point, however, you won’t have the ability to send more headers, so your only option would be to log the results into a file manually, which will be less efficient than having Nginx/Apache do the logging for you.

Both ways are much more difficult to implement, but will allow you to log the cache hits and misses without modifying any existing plugin code, so you wouldn’t have to worry about losing your changes to an update.

Logging The Headers

Now that you have the cache status headers in place, it’s time to log them to your web server access log. If you’ve ever messed around with your web server configuration files, you’ll know that this is not a difficult thing to do.

Here’s our example for Nginx, a variation of which we use on production nodes:

log_format cache-perf 'timestamp:$msec cache:$upstream_http_x_cache_status';
access_log /var/log/nginx/cache-perf.log cache-perf;

We’re not Apache fans/experts, but with the LogFormat and CustomLog directives, and the cache:%{X-Cache-Status}o placeholder, you should be able to get the data that you need.

Run this in production to see if it’s working:

$ tail -f /var/log/nginx/cache-perf.log
timestamp:1458649133.275 cache:hit
timestamp:1458649136.678 cache:miss
timestamp:1458649136.845 cache:-
timestamp:1458649138.392 cache:hit
timestamp:1458649139.784 cache:miss
timestamp:1458649141.098 cache:miss
...

That empty cache entry means the request was served without the cache status header, likely because it was a static file. If you’re planning to have this log running for more than a couple days, be sure to include it in your logrotate config.

Now that you’re collecting this data, give it a few hours in production and proceed to measuring your results.

Measuring the Results

There are various ways you can measure the results. If you want a quick snapshot of cache hits versus cache misses in the current log file, you can use some basic Linux command-line utilities, such as cat (or tac) and wc:

$ cat /var/log/nginx/cache-perf.log | grep cache:hit | wc -l
9210 # cache hits!
$ cat /var/log/nginx/cache-perf.log | grep cache:miss | wc -l
549 # cache misses!

If you want to look at a specific time frame, you can add some regular expressions to the mix:

$ cat /var/log/nginx/cache-perf.log | grep '^timestamp:145864\(0[89][0-9]\{2\}\|[1-7][0-9]\{3\}\|8000\)' | grep cache:hit
# Cache hits between 1458640800 and 1458648000
# In other words March 22nd 2016, 10:00-12:00

Regex can get pretty messy very quickly, so if you’re looking to dig into your data we recommend you use something like Logstash to pipe your cache performance files to Elasticsearch, and then visualize them with Kibana:

Monitoring the WordPress Cache Hit Rate in Kibana
Monitoring the WordPress Cache Hit Rate in Kibana

You may also want to add more data to the logs, like the requested URL, referrer, user agent, etc. This way you’ll be able to find out where your cache misses are, exclude things like /wp-admin/ or wp-cron.php from the results and so on.

Object Cache

Page caching is not the only cache available in WordPress. You may have heard of object caching which can run with a persistent storage backend, such as Memcached or Redis. We don’t recommend logging every object cache hit or miss in production, but looking at the Memcached or Redis stats will give you a general idea of how your cache is performing:

$ redis-cli info stats
evicted_keys:220383
keyspace_hits:1382223
keyspace_misses:2501380

The three most important metrics to watch are the cache hits, misses and the number of keys evicted. This information alone does not make much sense without knowing the timeframe, so we recommend running stats every five minutes and sending your data elsewhere for visualization.

Munin may be a good and lightweight option for this, and luckily there’s a Redis plugin for munin which works pretty much out of the box and supports authentication. If you prefer Memcached for your WordPress object cache, the munin-plugins-extra Debian package has a working memcached_ plugin available.

What’s a Good Cache Hit Rate?

A good cache hit rate depends a lot on your site. For simple blogs we consider a page cache hit-rate between 70% and 90% to be rather good. If all your cached data fits in memory and the eviction rate is low, you’ll want to be closer to the 90.

For very dynamic sites, e-commerce projects and sites with a lot of logged-in user activity, we’re seeing an average page cache hit rate of about 40-50%, though for sites that choose to put their cart items on every page, do price variations based on geo-location, initiate sessions early, etc., the cache hit rate may drop to anywhere between 0% and 30%. For these type of sites having a fast response time on cache misses is key for handling high traffic.

With regards to object caching, the hit rate should be over 90%, otherwise you’re simply wasting too many round-trips to Redis or Memcached, only to end up querying that data from MySQL. If you’re seeing a lower object cache hit rate, consider adjusting the memory allocated to your object cache, or decreasing the amount of data being stored until your eviction rate is closer to zero.

What’s Next?

Now that you’re monitoring your cache hit rate, keep an eye out on your graphs, especially when installing new WordPress themes/plugins or updates. If you see a sharp drop, chances are that the theme or plugin is doing something weird, like PHP sessions or cookies.

P.S. If you’re hosted with Pressjitsu, you already have access to some of these metrics (and more!) for your WordPress site via the Developer Console. If you’d like to dig in further, please open a support request and we’ll be happy to share more. If you’re not hosted with Pressjitsu, check out our features and plans.