Nginx¶

If you serve Django behind Nginx, then you can delegate the file streaming to Nginx and get increased performance:

lower resources used by Python/Django workers ;
faster download.

See Nginx X-accel documentation [1] for details.

Known limitations¶

Nginx needs access to the resource by URL (proxy) or path (location).
Thus VirtualFile and any generated files cannot be streamed by Nginx.

Given a view¶

Let’s consider the following view:

import os

from django.conf import settings
from django.core.files.storage import FileSystemStorage

from django_downloadview import StorageDownloadView

storage_dir = os.path.join(settings.MEDIA_ROOT, "nginx")
storage = FileSystemStorage(
    location=storage_dir, base_url="".join([settings.MEDIA_URL, "nginx/"])
)


optimized_by_middleware = StorageDownloadView.as_view(
    storage=storage, path="hello-world.txt"
)

What is important here is that the files will have an url property implemented by storage. Let’s setup an optimization rule based on that URL.

Note

It is generally easier to setup rules based on URL rather than based on name in filesystem. This is because path is generally relative to storage, whereas URL usually contains some storage identifier, i.e. it is easier to target a specific location by URL rather than by filesystem name.

Setup XAccelRedirect middlewares¶

Make sure django_downloadview.SmartDownloadMiddleware is in MIDDLEWARE of your Django settings.

Example:

    "django.middleware.common.CommonMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django_downloadview.SmartDownloadMiddleware",
]
# END middlewares

Then set django_downloadview.nginx.XAccelRedirectMiddleware as DOWNLOADVIEW_BACKEND:

# END backend

Then register as many DOWNLOADVIEW_RULES as you wish:

    {
        "source_url": "/media/nginx/",
        "destination_url": "/nginx-optimized-by-middleware/",
    },
]
# END rules

Each item in DOWNLOADVIEW_RULES is a dictionary of keyword arguments passed to the middleware factory. In the example above, we capture responses by source_url and convert them to internal redirects to destination_url.

class django_downloadview.nginx.middlewares.XAccelRedirectMiddleware(get_response=None, source_dir=None, source_url=None, destination_url=None, expires=None, with_buffering=None, limit_rate=None, media_root=None, media_url=None)¶

Bases: ProxiedDownloadMiddleware

Configurable middleware, for use in decorators or in global middlewares.

Standard Django middlewares are configured globally via settings. Instances of this class are to be configured individually. It makes it possible to use this class as the factory in django_downloadview.decorators.DownloadDecorator.

process_download_response(request, response)¶: Replace DownloadResponse instances by NginxDownloadResponse ones.

get_redirect_url(response)¶: Return redirect URL for file wrapped into response.

is_download_response(response)¶

Return True for DownloadResponse, except for “virtual” files.

This implementation cannot handle files that live in memory or which are to be dynamically iterated over. So, we capture only responses whose file attribute have either an URL or a file name.

process_response(request, response)¶: Call process_download_response() if response is download.

Per-view setup with x_accel_redirect decorator¶

Middlewares should be enough for most use cases, but you may want per-view configuration. For nginx, there is x_accel_redirect:

django_downloadview.nginx.decorators.x_accel_redirect(view_func, *args, **kwargs)¶

Apply XAccelRedirectMiddleware to view_func.

Proxies (*args, **kwargs) to middleware constructor.

As an example:

import os

from django.conf import settings
from django.core.files.storage import FileSystemStorage

from django_downloadview import StorageDownloadView
from django_downloadview.nginx import x_accel_redirect
)


optimized_by_decorator = x_accel_redirect(
    StorageDownloadView.as_view(storage=storage, path="hello-world.txt"),
    source_url=storage.base_url,
    destination_url="/nginx-optimized-by-decorator/",
)


def _modified_headers(request):
    view = StorageDownloadView.as_view(storage=storage, path="hello-world.txt")
    response = view(request)
    response["X-Test"] = "header"
    return response


modified_headers = x_accel_redirect(
    _modified_headers,
    source_url=storage.base_url,
    destination_url="/nginx-modified-headers/",
)

Test responses with assert_x_accel_redirect¶

Use assert_x_accel_redirect() function as a shortcut in your tests.

import os

from django.core.files.base import ContentFile
import django.test
from django.urls import reverse

from django_downloadview.nginx import assert_x_accel_redirect

from demoproject.nginx.views import storage, storage_dir


def setup_file():
    if not os.path.exists(storage_dir):
        os.makedirs(storage_dir)
    storage.save("hello-world.txt", ContentFile("Hello world!\n"))


class OptimizedByMiddlewareTestCase(django.test.TestCase):
    def test_response(self):
        """'nginx:optimized_by_middleware' returns X-Accel response."""
        setup_file()
        url = reverse("nginx:optimized_by_middleware")
        response = self.client.get(url)
        assert_x_accel_redirect(
            self,
            response,
            content_type="text/plain; charset=utf-8",
            charset="utf-8",
            basename="hello-world.txt",
            redirect_url="/nginx-optimized-by-middleware/hello-world.txt",
            expires=None,
            with_buffering=None,
            limit_rate=None,
        )


class OptimizedByDecoratorTestCase(django.test.TestCase):
    def test_response(self):
        """'nginx:optimized_by_decorator' returns X-Accel response."""
        setup_file()
        url = reverse("nginx:optimized_by_decorator")
        response = self.client.get(url)
        assert_x_accel_redirect(
            self,
            response,
            content_type="text/plain; charset=utf-8",
            charset="utf-8",
            basename="hello-world.txt",
            redirect_url="/nginx-optimized-by-decorator/hello-world.txt",
            expires=None,
            with_buffering=None,
            limit_rate=None,
        )


class ModifiedHeadersTestCase(django.test.TestCase):
    def test_response(self):
        """'nginx:modified_headers' returns X-Sendfile response."""
        setup_file()
        url = reverse("nginx:modified_headers")
        response = self.client.get(url)
        assert_x_accel_redirect(
            self,
            response,
            content_type="text/plain; charset=utf-8",
            charset="utf-8",
            basename="hello-world.txt",
            redirect_url="/nginx-modified-headers/hello-world.txt",
            expires=None,
            with_buffering=None,
            limit_rate=None,
        )
        self.assertEqual(response["X-Test"], "header")

django_downloadview.nginx.tests.assert_x_accel_redirect(test_case, response, **assertions)¶

Make test_case assert that response is a XAccelRedirectResponse.

Optional assertions dictionary can be used to check additional items:

basename: the basename of the file in the response.
content_type: the value of “Content-Type” header.
redirect_url: the value of “X-Accel-Redirect” header.
charset: the value of X-Accel-Charset header.
with_buffering: the value of X-Accel-Buffering header. If False, then makes sure that the header disables buffering. If None, then makes sure that the header is not set.
expires: the value of X-Accel-Expires header. If False, then makes sure that the header disables expiration. If None, then makes sure that the header is not set.
limit_rate: the value of X-Accel-Limit-Rate header. If False, then makes sure that the header disables limit rate. If None, then makes sure that the header is not set.

The tests above assert the Django part is OK. Now let’s configure nginx.

Setup Nginx¶

See Nginx X-accel documentation [1] for details.

Here is what you could have in /etc/nginx/sites-available/default:

charset utf-8;

# Django-powered service.
upstream frontend {
    server 127.0.0.1:8000 fail_timeout=0;
}

server {
    listen 80 default;

    # File-download proxy.
    #
    # Will serve /var/www/files/myfile.tar.gz when passed URI
    # like /optimized-download/myfile.tar.gz
    #
    # See http://wiki.nginx.org/X-accel
    # and https://django-downloadview.readthedocs.io
    #
    location /proxied-download {
        internal;
        # Location to files on disk.
        alias /var/www/files/;
    }

    # Proxy to Django-powered frontend.
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://frontend;
    }
}

… where specific configuration is the location /optimized-download section.

Note

/proxied-download has the internal flag, so this location is not available for the client, i.e. users are not able to download files via /optimized-download/<filename>.

Assert everything goes fine with healthchecks¶

Healthchecks are the best way to check the complete setup.

Common issues¶

`Unknown charset "utf-8" to override`¶

Add charset utf-8; in your nginx configuration file.

`open() "path/to/something" failed (2: No such file or directory)`¶

Check your settings.NGINX_DOWNLOAD_MIDDLEWARE_SOURCE_DIR in Django configuration VS alias in nginx configuration: in a standard configuration, they should be equal.

References

Nginx¶

Known limitations¶

Given a view¶

Setup XAccelRedirect middlewares¶

Per-view setup with x_accel_redirect decorator¶

Test responses with assert_x_accel_redirect¶

Setup Nginx¶

Assert everything goes fine with healthchecks¶

Common issues¶

`Unknown charset "utf-8" to override`¶

`open() "path/to/something" failed (2: No such file or directory)`¶

Table of Contents

Related Topics

This Page

Nginx¶

Known limitations¶

Given a view¶

Setup XAccelRedirect middlewares¶

Per-view setup with x_accel_redirect decorator¶

Test responses with assert_x_accel_redirect¶

Setup Nginx¶

Assert everything goes fine with healthchecks¶

Common issues¶

Unknown charset "utf-8" to override¶

open() "path/to/something" failed (2: No such file or directory)¶

`Unknown charset "utf-8" to override`¶

`open() "path/to/something" failed (2: No such file or directory)`¶