Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Ubuntu - Nginx - Cache static files on Nginx ======

Configure nginx to set the **Expires** HTTP header and the **max-age** directive of the **Cache-Control** HTTP header of static files (such as images, CSS and Javascript files) to a date in the future so that these files will be cached by your visitors' browsers.

This saves bandwidth and makes your web site appear faster (if a user visits your site for a second time, static files will be fetched from the browser cache).

----

===== Configuring nginx =====

The **Expires** HTTP header can be set with the help of the [[http://nginx.org/en/docs/http/ngx_http_headers_module.html#expires|expires]] directive which can be placed in inside //http {}, server {}, location {}//, or an if statement inside a //location {}// block.

Usually you will use it in a location block for your static files, e.g. as follows:

<file bash>
location ~*  \.(jpg|jpeg|png|gif|ico|css|js)$ {
   expires 365d;
}
</file>

In the above example, all //.jpg, .jpeg, .png, .gif, .ico, .css//, and //.js// files get an **Expires** header with a date 365 days in the future from the browser access time.

Therefore, you should make sure that the //location {}// block really only contain static files that can be cached by browsers.

----

===== Reload nginx =====

<code bash>
/etc/init.d/nginx reload
</code>

----

You can use the following time settings with the **expires** directive:

  * **off** makes that the **Expires** and **Cache-Control** headers will not be modified.
  * **epoch** sets the **Expires** header to 1 January, 1970 00:00:01 GMT.
  * **max** sets the **Expires** header to 31 December 2037 23:59:59 GMT, and the **Cache-Control** max-age to 10 years.
  * A time without an //@// prefix means an expiry time relative to the browser access time.  A negative time can be specified, which sets the **Cache-Control** header to **no-cache**.  Example: //expires 10d;// or //expires 14w3d;//
  * A time with an //@// prefix specifies an absolute time-of-day expiry, written in either the form Hh or Hh:Mm, where H ranges from 0 to 24, and M ranges from 0 to 59. Example: //expires @15:34;//

----

You can use the following time units:

  * ms: milliseconds
  * s: seconds
  * m: minutes
  * h: hours
  * d: days
  * w: weeks
  * M: months (30 days)
  * y: years (365 days)

Examples: 1h30m for one hour thirty minutes, 1y6M for one year and six months.

Also note that if you use a far future //Expires// header you have to change the component's filename whenever the component changes.

Therefore it's a good idea to version your files.

For example, if you have a file //javascript.js// and want to modify it, you should add a version number to the file name of the modified file (e.g. javascript-1.1.js) so that browsers have to download it.

If you don't change the file name, browsers will load the (old) file from their cache.

Instead of basing the //Expires// header on the access time of the browser (e.g. //expires 10d;//), you can also base it on the modification date of a file (<color red>please note that this works only for real files that are stored on the hard drive!</color>) by using the modified keyword which precedes the time:

<file bash>
expires modified 10d;
</file>

----

===== Configuring Cache-Control and Expires Headers =====

There are two caching control response headers: **Cache-Control** and **Expires**.

Cache-Control is the newer version, which has more options than Expires and is generally more useful if you want finer control over your caching behavior.

If these headers are set, they can tell the browser that the requested file can be kept locally for a certain amount of time (including forever) without requesting it again.

If the headers are not set, browsers will always request the file from the server, expecting either **200 OK** or **304 Not Modified** responses.

We can use the header module to set these HTTP headers.

The header module is a core Nginx module, which means it doesn't need to be installed separately to be used.

To add the header module, edit the default Nginx configuration file.

<code bash>
sudo vi /etc/nginx/sites-available/default
</code>

Find the server configuration block, which looks like this:


<file nginx /etc/nginx/sites-available/default>
. . .
# Default server configuration
#

server {
    listen 80 default_server;
    listen [::]:80 default_server;

. . .
</file>

Add the following two new sections here: one before the server block, to define how long to cache different file types, and one inside it, to set the caching headers appropriately.

<file nginx /etc/nginx/sites-available/default>
. . .
# Default server configuration
#

# Expires map
map $sent_http_content_type $expires {
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    expires $expires;
. . .
</file>

The section before the server block is a new map block which defines the mapping between the file type and how long that kind of file should be cached.

We're using several different settings in this map:

  * The default value is set to **off**, which will not add any caching control headers.  It's a safe bet for content we have no particular requirements on how the cache should work.

  * For **text/html**, we set the value to **epoch**.  This is a special value which results explicitly in no caching, which forces the browser to always ask if the website itself is up to date.

  * For **text/css** and **application/javascript**, which are stylesheets and Javascript files, we set the value to **max**.  This means the browser will cache these files for as long as it can, reducing the amount of requests considerably given that there are typically many of these files.

  * The last setting is for **~image/**, which is a regular expression that will match all file types containing **image/** in their MIME type name (like **image/jpg** and **image/png**).  Like stylesheets, there are often a lot of images on websites that can be safely cached, so we set this to **max** as well.

Inside the server block, the **expires** directive (a part of the headers module) sets the caching control headers.  It uses the value from the **$expires** variable set in the map. This way, the resulting headers will be different depending on the file type.

Save and close the file to exit.

To enable the new configuration, restart Nginx.

<code bash>
sudo systemctl restart nginx
</code>

Next, let's make sure our new configuration works.

----

===== Testing =====

To test if your configuration works, you can use the Network analysis function of the Developer tools in the Firefox Browser and access a static file through Firefox (e.g. an image).

In the Header output, you should now see an //Expires// header and a //Cache-Control// header with a //max-age// directive (//max-age// contains a value in seconds, for example 31536000 is one year in the future):

----

==== Creating Test Files ====

In this step, we will create several test files in the default Nginx directory.

We'll use these files later to check Nginx's default behavior and then to test that browser caching is working.

To make a decision about what kind of file is served over the network, Nginx does not analyze the file contents; that would be prohibitively slow.

Instead, it just looks up the file extension to determine the file's MIME type, which denotes the purpose of the file.

Because of this behavior, the content of our test files is irrelevant.

By naming the files appropriately, we can trick Nginx into thinking that, for example, one entirely empty file is an image and another is a stylesheet.

Create a file named **test.html** in the default Nginx directory using truncate.  This extension denotes that it's an HTML page.

<code bash>
sudo truncate -s 1k /var/www/html/test.html
</code>

Let's create a few more test files in the same manner: one jpg image file, one css stylesheet, and one js JavaScript file.

<code bash>
sudo truncate -s 1k /var/www/html/test.jpg
sudo truncate -s 1k /var/www/html/test.css
sudo truncate -s 1k /var/www/html/test.js
</code>

The next step is to check how Nginx behaves with respect to sending caching control headers on a fresh installation with the files we have just created.

----

==== Checking the Default Behavior ====

By default, all files will have the same default caching behavior.

To explore this, we'll use the HTML file we created earlier, but you can run these tests with any of the example files.

So, let's check if **test.html** is served with any information regarding how long the browser should cache the response.

The following command requests a file from our local Nginx server and shows the response headers.

<code bash>
curl -I http://localhost/test.html
</code>

You should see several HTTP response headers:

<code bash>
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Sat, 10 Sep 2016 13:12:26 GMT
Content-Type: text/html
Content-Length: 1024
Last-Modified: Sat, 10 Sep 2016 13:11:33 GMT
Connection: keep-alive
ETag: "57d40685-400"
Accept-Ranges: bytes
</code>

In the second to last line you can see the **ETag** header, which contains a unique identifier for this particular revision of the requested file.

If you execute the previous curl command repeatedly, you will see the exact same ETag value.

When using a web browser, the ETag value is stored and sent back to the server with the **If-None-Match** request header when the browser wants to request the same file again — for example, when refreshing the page.

We can simulate this on the command line with the following command.  Make sure you change the ETag value in this command to match the ETag value in your previous output.

<code bash>
curl -I -H 'If-None-Match: "57d40685-400"' http://localhost/test.html
</code>

The response will now be different:

<code bash>
HTTP/1.1 304 Not Modified
Server: nginx/1.10.0 (Ubuntu)
Date: Sat, 10 Sep 2016 13:20:31 GMT
Last-Modified: Sat, 10 Sep 2016 13:11:33 GMT
Connection: keep-alive
ETag: "57d40685-400"
</code>

This time, Nginx will respond with **304 Not Modified**.  It won't send the file over the network again; instead, it will tell the browser that it can reuse the file it already has downloaded locally.

This is useful because it reduces network traffic, but it's not quite good enough for achieving good caching performance.

The problem with ETag is that browser **always** sends a request to the server asking if it can reuse its cached file.  Even though the server responds with a 304 instead of sending the file again, it still takes time to make the request and receive the response.

In the next step, we will use the headers module to append caching control information.  This will make the browser to cache some files locally without explicitly asking the server if its fine to do so.

----

===== Testing Browser Caching =====

Execute the same request as before for the test HTML file.

<code bash>
curl -I http://localhost/test.html
</code>

This time the response will be different.  You should see two additional HTTP response headers:

<code bash>
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Sat, 10 Sep 2016 13:48:53 GMT
Content-Type: text/html
Content-Length: 1024
Last-Modified: Sat, 10 Sep 2016 13:11:33 GMT
Connection: keep-alive
ETag: "57d40685-400"
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
Accept-Ranges: bytes
</code>

The **Expires** header shows a date in the past and Cache-Control is set with no-cache, which tells the browser to always ask the server if there is a newer version of the file (using the ETag header, like before).

You'll see a difference response with the test image file.

<code bash>
curl -I http://localhost/test.jpg
</code>

result:

<code bash>
Nginx response headers
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Sat, 10 Sep 2016 13:50:41 GMT
Content-Type: image/jpeg
Content-Length: 1024
Last-Modified: Sat, 10 Sep 2016 13:11:36 GMT
Connection: keep-alive
ETag: "57d40688-400"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Accept-Ranges: bytes
</code>

In this case, Expires shows the date in the distant future, and Cache-Control contains max-age information, which tells the browser how long it can cache the file in seconds.

This tells the browser to cache the downloaded image for as long as it can, so any subsequent appearances of this image will use local cache and not send a request to the server at all.

The result should be similar for both test.js and test.css, as both JavaScript and stylesheet files are set with caching headers too.

This means the cache control headers have been configured properly and your website will benefit from the performance gain and less server requests due to browser caching.

You should customize the caching settings based on the content for your website, but the defaults in this article are a reasonable place to start.

----

===== Conclusion =====

The headers module can be used to add any arbitrary headers to the response, but properly setting caching control headers is one of its most useful applications.

It increases performance for the website users, especially on networks with higher latency, like mobile carrier networks.

It can also lead to better results on search engines that factor speed tests into their results. Setting browser caching headers is one of the major recommendations from Google's PageSpeed testing tools.

More detailed information about the headers module can be found in Nginx's official headers module documentation.

----

===== References =====

http://wiki.nginx.org/HttpHeadersModule