Today, we’re pleased to announce an exciting enhanced file storage system for our WordPress hosting clients.

This update will mean faster media loading and page speeds. It ensures we can put the highest amount of resource into serving webpages quickly and serving media quickly too.

What does ‘enhanced’ file storage mean?

We’ve spoken before about the importance of having static resources, like images, served from a content delivery network (CDN) to take strain off the main web server. Our enhanced storage system takes this concept one step further, by separating images and other media resources onto their own server too.

This is a storage and serving system, not a manipulation system like a web server. There’s a little less flexibility.

Because of that, we’re only moving older images and media to the new system. Images uploaded in the last 30 days are still available for cropping, etc — but those older images will be considered ‘set’ and will move over to the new storage system transparently.

I’m hosted with you. What does this mean for me?

There are no changes to the WordPress dashboard or your day-to-day usage of your site. Directories won’t change, links won’t change, everything stays the same on the front end.

There are some trade-offs for static assets (like images and PDFs) older than 30 days that we outline at the end of this post. This change will have no ‘everyday’ impact when managing your WordPress site.

The benefits are in better resource allocation, meaning faster pageloads, more resource availability, and pricing staying steady. It also paves the way for further innovations in the future.

I’m thinking about Performance Foundry. Do I need to do anything special?

No, our migration and performance teams will look after everything for you! Keep reading if you’d like to see more about what we’re doing, and why this can help your site. Click here to see our Managed WordPress service plans.

Why change?

We have been eyeing up a change like this for some time, and had already started the process in our labs and test sites. Adding more complexity to a system should never be taken lightly; but we’re seeing real benefits across the board.

As existing client sites grew, we were seeing increasing resource strain during our backup processes. Part of the issue was the number of files on the server, and we examined several approaches to remedy this. Enhanced storage gave us the best long-term architecture while also making pageloads faster: win- win!

The new backup system will act as a failover for serving media too; so we potentially have higher uptime if third-party infrastructure goes down.

So why don’t we roll out this to all media, instead of just the older resources? One main reason: flexibility.

Images and mp3s don’t really change much over time

It’s true: images don’t change much after you upload them! Except when they do.

As you upload your images to your WordPress app at Performance Foundry, we run them through a high-quality image compression system. We’re currently using one called Kraken.io, and we review the benefits and technology afresh every year. In the first month, you might decide to replace or re-edit some content based on user feedback, but after that, most site managers would tend to upload new media to replace out-of-date images rather than do a straight edit of an existing one.

Almost no media over a couple of months old ever gets touched by site managers across all our hosted clients! That’s why we’ve chosen to move things over after a 30-day period.

What’s the trade off?

Yes, all progress comes at a cost! When we move media to the dedicated server, we are currently losing a little bit of flexibility. The new object storage we’ll be using is just for storage — not editing or manipulation.

We’re aware of the following known issues:

  • The ability to crop and resize older media directly from the WordPress app disappears. Of course, we can still replace media, but it becomes a manual job for a server engineer, rather than a fully automated process from within WordPress.
  • If you delete an image from the media library, it will be deleted from the database, but not the file storage system. (i.e. it’ll still be available online until removed by a Performance Foundry engineer.)
  • Running optimisation software on images was always a server-side process; but this adds some complexity for our engineering team.
  • When you change themes, you may need to resize the image thumbnails. Resizing thumbnails can be done via a plugin, but we strongly recommend against that: it’s a process that’s much more efficiently handled on the server. Now, that’s going to need engineer input to move files from storage to the web server and back to storage again. This should always be done during staging and development, instead of on the production site, so should not impact any of our clients.

In each case, we need to pull media back to the main server, manipulate the file, then send it back again.

Apart from the manipulation issues above, the entire process is completely transparent to site managers and users. Because most users never edit older media, these trade-offs should not impact 99.9% of our clients at any time; and the 0.1% can simply drop a note to our support desk to have their request actioned. At the same time, everyone gets better site speeds!

But what about your on-the-fly image compression?

Publisher and Publisher Plus clients get the advantage of on-the-fly media compression. This change works perfectly alongside that. Foundation and e-commerce clients get the benefit of our one-time image compression through Kraken. This upgrade also works smoothly alongside that too.

I’m still reading… What does the tech look like?

Our current system looks like the diagram below, with one server resource powering a worldwide content delivery network. Most requests are driven through the CDN, with any ‘misses’ coming to the main web server to pick up the new or expired content. (Note that this is a simplified view of our architecture, but hopefully it is detailed enough.)

In a perfect world, a user would get a “hit” from the page cache and all the resources, like images, fonts, scripts and style information would load from the CDN. There’s very little processing work done on the processing server itself. Normally, there are 80-150 resources to load on a photo-heavy site, like a standard blog post with comments and some animated user interactions. (We try to reduce this as much as possible for you, using tools like Anvil and custom-coded themes.)

In the best-case scenario, pages are served to the user by the page cache, and assets from the CDN.

But when a cached copy of the page isn’t available, the request gets routed to the server. Now we have to process information from WordPress, using php and the database. (We have another level of database caching too, to accelerate that side of things.) But static resources are still served from the CDN. The page creation process can be really time consuming, taking anywhere from 0.1 to multiple seconds — that’s where leaner code comes into it!

If the page is new, or updated since the last cache hit, the user is directed to the web server to created the page. Assets are still served by the CDN.

There’s a perfect storm brewing when neither the page cache is available, nor the CDN. In this case we have both the page creation and the ~100 resources loading from the server. Each request for information burns some cycles and takes resources away from serving pages quickly.

This is our worst-case scenario: neither page caching nor the CDN is available, so the web server has to do all the work!

Our new enhanced file system has two resources sitting behind that CDN: one powering page generation and recent media, and one serving older media from a dedicated media server. The decision about where to send the request happens at the same control level as before, meaning no added latency.

With our new enhanced storage capacity, a CDN miss isn’t so bad! Assets older than 30 days are served from a dedicated media server.

The benefits are clear: more resources available to build webpages quickly, and a dedicated media server providing content to the CDN means faster service to the CDN and better handling of missed resources too.

In the case of a page cache hit and CDN miss, it’s now possible for the server to do very little work at all.

In this best-of-the-worst scenario, there’s a CDN miss; but only for assets older than 30 days. It bypasses processing on the main server, and gets those files from the media server.

What’s the deployment plan?

We’re going to start with our US clients: there’s one server that has been particularly impacted by the resource limitations, and that server has already been running a few sites with enhanced storage during our testing period. We’ll continue moving ahead with that server first.

This will have the biggest benefits for the biggest sites, so we’re rolling it out to our Plus clients, then Standard clients first, and then for our Foundation clients. After one server is settled in, we’ll move onto the next one.

Enterprise-level clients can opt-in through a discussion with their project manager.

Another step forward for Performance Foundry’s clients

We’re excited to be able to offer this enhanced file storage method to our clients — giving another dose of enterprise-level operations to our e-commerce operators and publishers. These innovations in file storage will ensure that all our client sites will continue to have a competitive advantage over their peers, serving their users better.