Introduction

Smart HTTP cache clearing

Smart HTTP cache clearing refers to the ability to clear cache for Locations/content that is in relation with the content being currently cleared.

When published, any Content item usually has at least one Location, identified by its URL. Therefore, HTTP cache being bound to URLs, if a Content item is updated (a new version is published), we want HTTP cache for all its Locations to be cleared, so the content itself can be updated everywhere it is supposed to be displayed. Sometimes, clearing cache for the content's Locations is not sufficient. You can, for instance, have an excerpt of it displayed in a list from the parent Location, or from within a relation. In this case, cache for the parent Location and/or the relation need to be cleared as well (at least if an ESI is not used).

The mechanism

Smart HTTP cache clearing is an event-based mechanism. Whenever a content item needs its cache cleared, the cache purger service sends an ezpublish.cache_clear.content event (also identified by eZ\Publish\Core\MVC\Symfony\MVCEvents::CACHE_CLEAR_CONTENT constant) and passes an eZ\Publish\Core\MVC\Symfony\Event\ContentCacheClearEvent event object. This object contains the ContentInfo object we need to clear the cache for. Every listener for this event can add Location objects to the cache clear list.

Once the event is dispatched, the purger passes collected Location objects to the purge client, which will effectively send the cache BAN request.

Default behavior

By default, following Locations will be added to the cache clear list:

• All Locations assigned to content (AssignedLocationsListener)
• Parent Location of all Content item's Locations (ParentLocationsListener)
• Locations for content's relations, including reverse relations (RelatedLocationsListener)

Implementing a custom listener

By design, smart HTTP cache clearing is extensible. One can easily implement an event listener/subscriber to the ezpublish.cache_clear.content event and add Locations to the cache clear list.

Example

Here's a very simple custom listener example, adding an arbitrary Location to the list.

Content Cache

eZ Platform uses Symfony HttpCache to manage content "view" cache with an expiration model. In addition it is extended (using FOSHttpCache) to add several advanced features. For content coming from the CMS the following is taken advantage of out of the box:

• To be able to always keep cache up to date, cache is made "content-aware" to allow updates to content to trigger cache invalidation.
  • Uses a custom X-Location-Id header, which both Symfony and Varnish Proxy are able to invalidate cache on (for details see Cache purge.)
• To be able to also cache requests by logged-in users, cache is made "context-aware ."
  • Uses a custom vary header X-User-Hash to allow pages to var by user rights (so not per unique user, that is better served by browser cache.)

Cache and Expiration Configuration

Making your controller response content-aware

Sometimes you need your controller's cache to be invalidated at the same time as specific content changes (i.e. ESI sub-requests with render twig helper, for a menu for instance). To be able to do that, you just need to add X-Location-Id header to the response object:

Making your controller response context-aware

If the content you're rendering depends on a user's permissions, then you should make the response context-aware:

Configuration

Cache Purge

This page explains the content cache purge (aka invalidate) mechanism used when publishing content from the UI or from a container-aware script, resulting in cache being invalidated either in the built-in Symfony Reverse Proxy, or on the much faster Varnish reverse proxy.

Overview

eZ Platform returns content-related responses with an X-Location-Id header that are stored together by the configured HTTP cache. This allows you to clear (invalidate) HTTP cache representing specifically a given Content item. On publishing the content, a cache purger is triggered with the Content ID in question, which in turn figures out affected content Locations based on Smart HTTP cache clearing logic. The returned Location IDs are sent for purge using the purge type explained further below.

Purge types

Symfony Proxy: Local purge type

By default, invalidation requests will be emulated and sent to the Symfony Proxy cache Store. Cache purge will thus be synchronous, meaning no HTTP purge requests will be sent around when publishing.

Varnish: HTTP purge type

With Varnish you can configure one or several servers that should be purged over HTTP. This purge type is asynchronous, and flushed by the end of Symfony kernel-request/console cycle (during terminate event). Settings for purge servers can be configured per site group or site access:

For further information on setting up Varnish, see Using Varnish.

Purging

While purging on content, updates are handled for you; on actions against the eZ Platform APIs, there are times you might have to purge manually.

Manual by code

Manual purging from code which takes Smart HTTP cache clearing logic into account, this is using the service also used internally for cache clearing on content updates:

Manually by command with Symfony Proxy

Symfony Proxy stores its cache in the Symfony cache directory, so a regular cache:clear commands will clear it:

Manual by HTTP BAN request on Varnish

When using Varnish and in need to purge content directly, then the following examples show how this is done internally by our FOSPurgeClient, and in turn FOSHttpCache Varnish proxy client:

For purging all:

Or with given location ids (here 123 and 234): 

Using Varnish

eZ Platform being built on top of Symfony, it uses standard HTTP cache headers. By default the Symfony reverse proxy, written in PHP, is used to handle cache, but it can be easily replaced with any other reverse proxy like Varnish.

Use of Varnish is a requirement for use in Clustering setup, for overview of clustering feature see Clustering. 

Prerequisites

• A working Varnish 3 or Varnish 4 setup.

Recommended VCL base files

For Varnish to work properly with eZ, you'll need to use one of the provided files as a basis:

• Varnish 3 VCL example
• Varnish 4 VCL example

Note: Http cache management is done with the help of FOSHttpCacheBundle. You may need to tweak your VCL further on according to FOSHttpCache documentation in order to use features supported by it.

Configure eZ Publish

Update your Virtual Host

Somehow we need to tell php process that we are behind a Varnish proxy and not the built in Symfony Http Proxy. If you use fastcgi/fpm you can pass these directly to php process, but you can in all cases also specify them in your web server config.

On apache:

On nginx:

Update YML configuration

Secondly we need to tell eZ Platform to change to use http based purge client (specifically FosHttpCache Varnish purge client is used), and specify url Varnish can be reached on:

Usage

Context-aware HTTP cache

Use case

As it is based on Symfony 2, eZ Platform uses HTTP cache extended with features like content awareness. However, this cache management is only available for anonymous users due to HTTP restrictions.

It is of course possible to make HTTP cache vary thanks to the Vary response header, but this header can only be based on one of the request headers (e.g. Accept-Encoding). Thus, to make the cache vary on a specific context (for example a hash based on a user roles and limitations), this context must be present in the original request.

Feature

As the response can vary on a request header, the base solution is to make the kernel do a sub-request in order to retrieve the user context hash (aka user hash). Once the user hash has been retrieved, it's injected in the original request in the X-User-Hash custom header, making it possible to vary the HTTP response on this header:

This solution is implemented in Symfony reverse proxy (aka HttpCache) and is also accessible to dedicated reverse proxies like Varnish.

HTTP cache clear

As eZ Platform uses FOSHttpCacheBundle, this impacts the following features:

• HTTP cache purge
• User context hash

Varnish proxy client from FOSHttpCache lib is used for clearing eZ HTTP cache, even when using Symfony HttpCache. A single BAN request is sent to registered purge servers, containing a X-Location-Id header. This header contains all Location IDs for which objects in cache need to be cleared.

Symfony reverse proxy

Symfony reverse proxy (aka HttpCache) is supported out of the box, all you have to do is to activate it.

Varnish

User context hash

FOSHttpCacheBundle User Context feature is activated by default.

As the response can vary on a request header, the base solution is to make the kernel do a sub-request in order to retrieve the context (aka user context hash). Once the user hash has been retrieved, it's injected in the original request in the X-User-Hash header, making it possible to vary the HTTP response on this header:

This solution is implemented in Symfony reverse proxy (aka HttpCache) and is also accessible to dedicated reverse proxies like Varnish.

Workflow

User hash generation

eZ Platform already interferes with the hash generation process by adding current user permissions and limitations. You can also interfere in this process by implementing custom context provider(s).

User hash generation with Varnish 3

The behavior described here comes out of the box with Symfony reverse proxy, but it's of course possible to use Varnish to achieve the same.

User hash generation with Varnish 4

New anonymous X-User-Hash

The anonymous X-User-Hash is generated based on the anonymous user, group and role. The 38015b703d82206ebc01d17a39c727e5 hash that is provided in the code above will work only when these three variables are left unchanged. Once you change the default permissions and settings, the X-User-Hash will change and Varnish won't be able to effectively handle cache anymore.

In that case you need to find out the new anonymous X-User-Hash and change the VCL accordingly, else Varnish will return a no-cache header.

The easiest way to find the new hash is:

1. Connect to your server (shh should be enough)

2. Add <your-domain.com> to your /etc/hosts file

3. Execute the following command:

curl -I -H "Accept: application/vnd.fos.user-context-hash" http://<your-domain.com>/_fos_user_context_hash

You should get a result like this:

4. Now, replace the existing X-User-Hash value with the new one:

5. Restart the Varnish server and everything should work fine.

Default options for FOSHttpCacheBundle defined in eZ

The following configuration is defined in eZ by default for FOSHttpCacheBundle. You may override these settings.