Cacheing

Note: This doesn’t really sit directly in a database book but it’s too small a topic to be turned into a book in its own right.

What is a cache?

The Oxford English Dictionary (OED) defines a cache like this:

Noun: a collection of items of the same type stored in a hidden or inaccessible place:
an arms cache
a cache of gold coins

More specifically for computing:

an auxilliary memory from which high-speed retrieval is possible

It goes on to explain why you might want to do this, saying store away for future use.

The last part of the dictionary definition gives us a clue here. If something is difficult or time-consuming to create then why go through the creation process again? Instead just keep a copy of the thing. In other cases, if we need something over and over again, why not just create it once and then reuse it?

The problem with cacheing

Doing things once and keeping hold of the result works really well, and can speed things up no end. What happens when the underlying data changes, though? You have stale data and may make bad decisions or just not be able to see what you expect to see.

There needs to be a process that lets you invalidate the cache. This means it will be reconstructed when you next ask for the information.

Most cacheing technology also tends to use a store that forgets things after a given amount of time. This results in the re-creation of cached objects every so often anyway. It’s a way of making sure things stay current even if there is no direct invalidation. Sometimes for performance reasons tolerating a degree of staleness is a way of optimising response times. As usual there’s a trade off.

The Russian doll problem

You may be familiar with the Matryoshka doll which is a set of dolls that each contain smaller and smaller dolls beloved of tourists that go to Russia. There is a similar problem with cacheing. You may have something like an HTML view that is comprised of smaller view fragments. Those fragments can themselves be comprised of smaller fragments, and so on. Imagine you have cached all of these fragments in your data store.

Let’s think this through:

The data behind one of the fragments changes.
A very expensive way of solving this is to machine gun all of the cached objects and reconstruct the whole view from scratch - we don’t want to do this if 90% of what we will send back is the same.
Rails has a cacheing system uses a key usually based on the last updated time of the object being rendered.
As the view is reconstructed the time stamped keys are checked against what has been stored.
If it has changed that identifies that cached object needs re-rendering.
As the view is rendered it only re-creates and re-stores (as in stores again) the pieces that have changed. So the 90% of the fragments that are unchanged are not re-rendered.

Storing and invalidating cached objects is one of the hardest problems in computer science because identifying changes without having to re-load all the data is hard. Russian doll cacheing is a pragmatic approach to this and it works well. It has been in Rails since Rails 5.

Other view cacheing strategies

In the days when the web wasn’t as fast as it is now many sites wanted to show their customers something straight away inside the magic 100 ms window. Showing the sexy page with all of the dynamic menus and stuff took a long time to arrive at the browser. So instead they would show a page full of placeholders that populated itself using Javascript after it arrived. This eventually mutated into the single page app that ironically these days locks up and takes 10 minutes to download all its Javascript libraries and images. Doing a simple version of this can still work really well. Rails’ turbo library is an attempt to do this transparently and is turned on by default in new apps.

I have a memory of a building society that had a very clever Flash animation showing how using their services meant you didn’t have to worry about security. It had an animation of some kind of private eye character who had heavy footsteps. You would have to wait several minutes for the animation to load while the heavy footsteps kept coming out of the speakers of your laptop. After it loaded you could close it and interact with the site proper. This is obviously the opposite of this strategy but it was in the days when not very many folks were on the internet so probably forgivable, at least they were trying something to make them stand out, albeit perhaps not how they intended.

APIs

Another win can be to turn the arguments you’re sent for a query API into some kind of key or hash and store the results for a while in a cache store like Redis. If you get a hit for your hash in Redis return that instead. Again, the store needs to be set to forget things after a while, but it can usefully speed up response times if used carefully.

Where do we store our cached stuff?

There are two main places to do this:

Files. Store the rendered view fragments in a temporary directory using the cache keys a file names. This works well if you have one server, unless you’re willing to have the temporary directory on every node in your front end and build things on demand per server. This could work.
Key value stores. Instead use a key-value system like Redis or perhaps CouchDB. This allows you to store any data you like against a key you specify. The service provided means you can share it across multiple servers. There will of course be overhead using such a service, so it has to be quicker than regenerating the things you are cacheing. Redis can also do things like set a lifetime for a given item in the store, using files means you would have to do this yourself or use a system that does it for you.

Let’s look at some code

Here we’re just talking about cacheing view rendering. We’re going to look at the use of the view cache helper. If you search for the work cache in the Rails API there it appears in a lot of places.

At its most basic code looks like this:

<customer>
  <h1><%= customer.name %></h1>	
    <ul>
      <% customer.orders.each do |order| %>
        <% cache(order) do %>
          <%= render order %>
        <% end %>
     <% end %>
  </ul>
</customer>

So what are we doing here? The cache helper creates a key based on the order’s class, key, and last updated date. The first time it is called it creates an entry in the cache store by calling the block, for subsequent calls it checks the cache key and returns what’s stored if they match, otherwise it calls the block again and creates a new entry for the new key. This is also why things need to eventually disappear from the cache, there’s no point in keeping data for invalidated keys around forever, and tracking when something was last accessed is much harder work that just letting them expire over time.

This code may have a hidden problem though.

Say within the order rendering we have order lines being rendered:

<order>
  <h1>Order on: <%= order.date %></h1>
  <% order.order_lines.each do |line| %>
	<%= render line %>
  <% end %>
</order>

We haven’t cached each line (although we could have). So what is the problem?

If the order line changes it won’t invalidate the cached order view. We also need to change the order line model:

class OrderLine
  belongs_to :order, touch: true
end

If an order line changes it will touch the updated at of its parent order and invalidate the view cache when its next rendered.

You can also just call touch on any ActiveRecord object and it will do this, so in situations where the automatic handling of touch with this belongs_to aren’t enough you can still make it happen directly.

This also means that if we were to cache the order lines only the one that changed would be regenerated, reusing the cached order lines where needed.

Database: a guide for the perplexed