Don’t Cache Everything in a Transient

The Transients API is a great way to cache small pieces of data in WordPress, but there are certain things developers tend to overlook when working with this API. In this post we’ll cover some situations where transient caching is not a good fit, and explore some better alternatives.

The WordPress Transients API

The Transients API

We’re not going to cover the basics of the Transients API in this post, if you haven’t worked with it before, this codex page will get you started. As a developer, there are a few things you should know and keep in mind when working with transients in WordPress:

  • A transient is volatile, and is not guaranteed to exist
  • A transient is stored in wp_options or a persistent object caching backend if present
  • When stored in wp_options, transients are not autoloaded by default
  • Transients don’t expire on their own
  • When a transient is expired, it’s expired
  • Transients are not thread-safe

To overcome some of these limitations, numerous plugins, libraries and code snippets have emerged, for example: Mark Jaquith’s TLC Transients library allows developers to serve stale data while regenerating it in the background, our Transients Cleaner plugin removes expired transients from the database on a daily basis, a persistent Redis object-cache backend with a no-eviction policy can guarantee a transient’s existence for a set period of time, sort of.

But another important problem is that given the simplicity of the Transients API, many developers choose to use it for anything they can possibly think of: remote requests, navigation menus, share and followers counts, stats, comments, widgets and much, much more. Some of them can be great use cases, but others not so much.

Let’s look at a few examples.

Things You Shouldn’t Cache in a Transient

Share Counts

Including likes, tweets, retweets, diggs, upboats and everything else. These numbers are usually obtained through remote HTTP requests, and while the Transients API is a really good way of caching such requests, you really don’t want to be saving those numbers to a volatile storage or in your options table.

With a volatile storage (Memcached, Redis with LRU, etc.) you may lose these share counts every once in a while, and retrieving them again may become quite expensive, and even more so if you run out of memory.

Share Counts in WordPress
Share Counts in WordPress

Storing share counts in your options table is not going to scale — 100k posts = 200k wp_options rows for transients = bad business, in addition to the two extra SQL queries you’re going to perform per post, to display your share count. That’s 40 additional queries on an archive page with twenty posts. If you’re unlucky, those twenty transients may have expired.

Navigation Menus

In general caching calls to wp_nav_menu() is a great idea, and if you can cache such a menu under a single cache key, then a transient is a really good choice.

But sooner or later you’ll notice that the output of wp_nav_menu() depends a lot on the current context, all those current-menu-item classes, child/parent classes, etc., are often used in themes to highlight items in the menu.

Context in WordPress Navigation Menus
Context in WordPress Navigation Menus

To retain this behavior, you’re going to have to somehow retain the context, make it part of the transient cache key, and before you know it, you end up with thousands of transient entries, one for each post, page, year, month, day.

This Menu Cache plugin goes even further and uses the current REQUEST_URI to generate the transient cache key, which is a really bad idea, not only because you’re wasting memory on every single search query, or a newsletter campaign with unique per-subscriber ?utm_ variables, but also because it can easily be abused.

If you’re going to cache a navigation menu in a transient, make sure you’re varying by very little context, such as current language or logged in vs. logged out. All the hover/active bells and whistles can then be added on the fly with JavaScript.

Comments

Rendering comments in WordPress is usually pretty quick, unless you’re dealing with a large number of comments on a single page. You’ll start noticing the slowness at a few hundred, and after looking at the script execution profile, you’ll see that it’s not the retrieval of the comments from the database that’s most expensive, but actually printing them out on the screen — running them through the various filters, converting smileys to images and things like that.

Core supports breaking the comments down by pages, but some SEO experts will argue against that. JavaScript-based third-party commenting systems can render comments very efficiently without querying your server, but some SEO exports will argue against that.

In any case, caching these comments with the Transients API may come to mind, and seem like a good idea at first, but again, do you really want to store the full comment thread for every single post and page in your options table? No. Do you want to perform a write operation on the wp_options table and invalidate its internal MySQL caches, every time somebody posts a new comment? Nope.

Other Examples

The examples above are just a few cases where the Transients API won’t do a good job, especially at scale. You can probably think of many other scenarios where the data doesn’t really belong or “fit” in wp_options, or in a volatile in-memory cache, even though implementing that with a transient would be a breeze, so let’s look at some alternative solutions.

Metadata API as an Alternative

The most overlooked alternative to transient caching is the Metadata API, and you’re right, it’s not a caching API. But just like the Transients API sits on top of options in WordPress, some simple code can do that for metadata, here’s an example:

$cache_key = '_my-cache-key';
$cache = get_post_meta( $post->ID, $cache_key, true );
if ( empty( $cache ) || $cache['expires'] < time() )
    $data = something_expensive();
    $cache = array(
        'expires' => time() + 2 * HOUR_IN_SECONDS,
        'data' => $data,
    );

    update_post_meta( $post->ID, $cache_key, $cache );
}

echo $cache['data']; // cached or uncached result

Here we run something_expensive() once every two hours, and cache it in the metadata for the given post. This is great for several reasons:

  • The stored data is not volatile, even if it’s “expired”
  • If something_expensive() is unavailable at this time, we can keep the old value
  • When fetching with WP_Query, all meta caches are primed by default, so fetching our key will not cause any additional trips to MySQL, Redis, Memcached, etc.

By wrapping this into a custom function, a class, or a library, you can adapt it and re-use it in every project of yours. We did that with our Fragment Caching class, which reads the output buffer and supports caching in metadata, transients and object cache. We use it for some of our customers to cache navigation menus, widgets, long comment threads and more.

Keep it Under Control

The disadvantage of this approach is that you have to keep track of your cached data. With transients it’s fine to change or randomize the cache key, especially with persistent object caching which will eventually just evict the old entry.

But with metadata it’s different, because the cached data is now loaded into memory whenever there’s a request for any metadata for that particular post. This means you should never use anything random or timestamp-y for cache keys.

Looking through all unique meta keys from time to time is also a good idea:

SELECT meta_key, COUNT(*) FROM wp_postmeta GROUP BY meta_key;

+----------------------------------+----------+
| meta_key                         | count(*) |
+----------------------------------+----------+
| _edit_last                       |       41 |
| _edit_lock                       |       70 |
| _menu_item_classes               |        5 |
| _menu_item_menu_item_parent      |        5 |
| _menu_item_object                |        5 |
...

Not Just for Posts

In addition to posts, the Metadata API in WordPress spans across comments, users and even terms (since 4.4). So if you’re looking for a place to store the number of Twitter followers for each one of your authors, the wp_usermeta table is a great place. Number of up-votes on a comment – wp_commentmeta. You get the idea…

Other Alternatives

Other alternatives to transients in WordPress are options, object cache and custom tables. With options you can use a very similar approach to the one we’ve shown above for metadata.

For example, if you’d like to display the number of RSS subscribers in your sidebar via some third party API, then an option may be a better choice than a transient, because you can show a stale number if that API is down, and the option will be autoloaded by default during wp_load_alloptions().

As for object caching (wp_cache_*), you’ll rarely want to use these functions directly, but if you do, make sure you understand what persistency is and how cache groups work. If you’re simply looking for a way to store some data and reuse it later in the same request, perhaps a static variable or a class variable is a better choice.

One last alternative worth mentioning is a custom database table, and you’ll only want to use this if you need absolute control.

Wrapping Up

Transients are definitely one of the easiest ways to cache things in WordPress, but before you do put something in a transient, make sure you have good reasons to do so. As we’ve shown above, some things are better cached in other places, such as metadata tables or even custom tables.

Keep a close eye on what you’re caching, under what cache keys, in which database tables and the size of your cached data. Make sure you know when exactly the data is being retrieved from the database – something required on a single page but loaded on all of them is a waste of time and memory.

Subscribe to our free newsletter for tips about WordPress development, performance, scaling, and security.