Lighttpd

If you serve Django behind Lighttpd, then you can delegate the file streaming to Lighttpd and get increased performance:

  • lower resources used by Python/Django workers ;
  • faster download.

See Lighttpd X-Sendfile documentation [1] for details.

Note

Currently, django_downloadview supports X-Sendfile, but not X-Sendfile2. If you need X-Sendfile2 or know how to handle it, check X-Sendfile2 feature request on django_downloadview’s bugtracker [2].

Known limitations

  • Lighttpd needs access to the resource by path on local filesystem.
  • Thus only files that live on local filesystem can be streamed by Lighttpd.

Given a view

Let’s consider the following view:

import os

from django.conf import settings
from django.core.files.storage import FileSystemStorage

from django_downloadview import StorageDownloadView


storage_dir = os.path.join(settings.MEDIA_ROOT, 'lighttpd')
storage = FileSystemStorage(
    location=storage_dir,
    base_url=''.join([settings.MEDIA_URL, 'lighttpd/']))


optimized_by_middleware = StorageDownloadView.as_view(storage=storage,
                                                      path='hello-world.txt')

What is important here is that the files will have an url property implemented by storage. Let’s setup an optimization rule based on that URL.

Note

It is generally easier to setup rules based on URL rather than based on name in filesystem. This is because path is generally relative to storage, whereas URL usually contains some storage identifier, i.e. it is easier to target a specific location by URL rather than by filesystem name.

Setup XSendfile middlewares

Make sure django_downloadview.SmartDownloadMiddleware is in MIDDLEWARE_CLASSES of your Django settings.

Example:

    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django_downloadview.SmartDownloadMiddleware'
]
# END middlewares

Then set django_downloadview.lighttpd.XSendfileMiddleware as DOWNLOADVIEW_BACKEND:

"""Could also be:

Then register as many DOWNLOADVIEW_RULES as you wish:

DOWNLOADVIEW_BACKEND = 'django_downloadview.lighttpd.XSendfileMiddleware'
        'destination_dir': '/apache-optimized-by-middleware/',
        # Bypass global default backend with additional argument "backend".
        # Notice that in general use case, ``DOWNLOADVIEW_BACKEND`` should be
        'source_url': '/media/lighttpd/',
        'destination_dir': '/lighttpd-optimized-by-middleware/',

Each item in DOWNLOADVIEW_RULES is a dictionary of keyword arguments passed to the middleware factory. In the example above, we capture responses by source_url and convert them to internal redirects to destination_dir.

class django_downloadview.lighttpd.middlewares.XSendfileMiddleware(source_dir=None, source_url=None, destination_dir=None)

Bases: django_downloadview.middlewares.ProxiedDownloadMiddleware

Configurable middleware, for use in decorators or in global middlewares.

Standard Django middlewares are configured globally via settings. Instances of this class are to be configured individually. It makes it possible to use this class as the factory in django_downloadview.decorators.DownloadDecorator.

process_download_response(request, response)

Replace DownloadResponse instances by XSendfileResponse ones.

get_redirect_url(response)

Return redirect URL for file wrapped into response.

is_download_response(response)

Return True for DownloadResponse, except for “virtual” files.

This implementation cannot handle files that live in memory or which are to be dynamically iterated over. So, we capture only responses whose file attribute have either an URL or a file name.

process_response(request, response)

Call process_download_response() if response is download.

Per-view setup with x_sendfile decorator

Middlewares should be enough for most use cases, but you may want per-view configuration. For Lighttpd, there is x_sendfile:

django_downloadview.lighttpd.decorators.x_sendfile(view_func, *args, **kwargs)

Apply XSendfileMiddleware to view_func.

Proxies (*args, **kwargs) to middleware constructor.

As an example:

import os

from django.conf import settings
from django.core.files.storage import FileSystemStorage

from django_downloadview import StorageDownloadView
from django_downloadview.lighttpd import x_sendfile


optimized_by_decorator = x_sendfile(
    StorageDownloadView.as_view(storage=storage, path='hello-world.txt'),
    source_url=storage.base_url,
    destination_dir='/lighttpd-optimized-by-decorator/')

Test responses with assert_x_sendfile

Use assert_x_sendfile() function as a shortcut in your tests.

import os

from django.core.files.base import ContentFile
from django.core.urlresolvers import reverse
import django.test

from django_downloadview.lighttpd import assert_x_sendfile

from demoproject.lighttpd.views import storage, storage_dir


def setup_file():
    if not os.path.exists(storage_dir):
        os.makedirs(storage_dir)
    storage.save('hello-world.txt', ContentFile(u'Hello world!\n'))


class OptimizedByMiddlewareTestCase(django.test.TestCase):
    def test_response(self):
        """'lighttpd:optimized_by_middleware' returns X-Sendfile response."""
        setup_file()
        url = reverse('lighttpd:optimized_by_middleware')
        response = self.client.get(url)
        assert_x_sendfile(
            self,
            response,
            content_type="text/plain; charset=utf-8",
            basename="hello-world.txt",
            file_path="/lighttpd-optimized-by-middleware/hello-world.txt")


class OptimizedByDecoratorTestCase(django.test.TestCase):
    def test_response(self):
        """'lighttpd:optimized_by_decorator' returns X-Sendfile response."""
        setup_file()
        url = reverse('lighttpd:optimized_by_decorator')
        response = self.client.get(url)
        assert_x_sendfile(
            self,
            response,
            content_type="text/plain; charset=utf-8",
            basename="hello-world.txt",
            file_path="/lighttpd-optimized-by-decorator/hello-world.txt")
django_downloadview.lighttpd.tests.assert_x_sendfile(test_case, response, **assertions)

Make test_case assert that response is a XSendfileResponse.

Optional assertions dictionary can be used to check additional items:

  • basename: the basename of the file in the response.
  • content_type: the value of “Content-Type” header.
  • file_path: the value of “X-Sendfile” header.

The tests above assert the Django part is OK. Now let’s configure Lighttpd.

Setup Lighttpd

See Lighttpd X-Sendfile documentation [1] for details.

Assert everything goes fine with healthchecks

Healthchecks are the best way to check the complete setup.

References

[1](1, 2) http://redmine.lighttpd.net/projects/lighttpd/wiki/X-LIGHTTPD-send-file
[2]https://github.com/benoitbryon/django-downloadview/issues/67