How to deploy Django project to Heroku using Docker

Django Heroku Tutorial Series:

Heroku vs AWS Which is Best for Your Django project

How to deploy Django project to Heroku using Docker

How to deploy Python project to Heroku in Gitlab CI

How to use Heroku Pipeline

Heroku Logs Tutorial

How to monitor Heroku Postgres using heroku-pg-extras

Django Dokku Tutorial Series:

How to deploy Django project to Dokku

How to deploy Django project to Dokku with Docker

Introduction

In this Django Heroku guide, I will talk about how to deploy Django project to Heroku using Docker.

So after you read it, you will get:

The difference between Heroku Buildpacks and Heroku Container.
How to serve Django static assets and media files in Heroku.
How to test Docker image for Django project in local env.
How to build front-end stuff for Django project when deploying.

The source code of this tutorial is django-heroku. I would appreciate that if you could give it a star.

Heroku Buildpacks and Heroku Container

There are mainly two ways for you to deploy Django project to Heroku

One way is the buildpacks.

Buildpacks are responsible for transforming deployed code into a slug, which can then be executed on a dyno

You can see Buildpacks as some pre-defined scripts maintained by Heroku team which deploy your Django project. They usually depend on your programming languages.

Buildpacks is very easy to use so most Heroku tutorial would like to talk about it.

Another way is using Docker.

Docker provides us a more flexible way so we can take more control, you can install any packages as you like to the OS, or also run any commands during the deployment process.

For example, if your Django project has frontend app which help compile SCSS and ES6, you want to npm ci some dependency packages during the deployment process, and then run npm build command, Docker seems more clean solution while buildpacks can do this.

What is more, Docker let you deploy project in a way most platforms can support, it can save time if you want to migrate it to other platforms such as AWS, Azure in the future.

Build manifest and Container Registry

Some people are confused about Build manifest and Container Registry in Heroku Docker doc. So let me explain here.

Container Registry means you build docker in local, and then push the image to Heroku Container Registry. Heroku would use the image to create a container to host your Django project.

Build manifest means you push Dockerfile and Heorku would build it, run it in standard release flow.

What is more, Build manifest support some useful Heorku built-in features such as Review Apps, Release.

If you have no special reason, I strongly recommend using Build Manifest way to deploy Django.

Docker Build vs Docker Run

So what is the difference between Docker build and Docker run

Docker build builds Docker images from a Dockerfile.

A Dockerfile is a text document that contains all the commands a user could call on the command line to build an image.

Docker run create a writeable container layer over the specified image,

So we should first use docker build to build the docker image from Dockerfile and then create container over the image.

Environment

First, let's add django-environ to requirements.txt.

django-environ==0.8.1

And then, let's update Django settings django_heroku/settings.py to use django-environ to read the env.

import environ                           # new
from pathlib import Path

env = environ.Env()                      # new


# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env('DJANGO_SECRET_KEY', default='django-insecure-$lko+#jpt#ehi5=ms9(6s%&6fsg%r2ag2xu_2zj1ibsj$pckud')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env.bool("DJANGO_DEBUG", True)

ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=[])

Notes:

Here we use django-environ to set SECRET_KEY, DEBUG and ALLOWED_HOSTS from the Environment variable.
And we also set default value to make it work in development without extra Env.

$ ./manage.py migrate
$ ./manage.py runserver

Now, please check on http://127.0.0.1:8000/ to make sure everyting is working.

.dockerignore file

Before the docker CLI sends the context to the docker daemon, it looks for a file named .dockerignore in the root directory of the context. If this file exists, the CLI modifies the context to exclude files and directories that match patterns in it. This helps to avoid unnecessarily sending large or sensitive files and directories to the daemon and potentially adding them to images using ADD or COPY

Let's create .dockerignore

**/node_modules
**/build

Sometimes, if your Django project is using NPM as frontend solution, we should NOT add files in node_modules to the Docker container.

We can use .dockerignore, so the files in node_modules and build directory would NOT be processed by Docker when ADD or COPY

Start to write Dockerfile

Now you already have basic understanding of the Heroku docker, so now let's learn more about Dockerfile,

Before we start, let's take a look at docker multi-stage builds

With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image.

Here I would use django_heroku as an example to show you how to write Dockerfile.

# Please remember to rename django_heroku to your project directory name
FROM node:14-stretch-slim as frontend-builder

WORKDIR /app/frontend

COPY ./frontend/package.json /app/frontend
COPY ./frontend/package-lock.json /app/frontend

ENV PATH ./node_modules/.bin/:$PATH

RUN npm ci

COPY ./frontend .

RUN npm run build

#################################################################################
FROM python:3.10-slim-buster

WORKDIR /app

ENV PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app \
    DJANGO_SETTINGS_MODULE=django_heroku.settings \
    PORT=8000 \
    WEB_CONCURRENCY=3

# Install system packages required by Wagtail and Django.
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends \
    build-essential curl \
    libpq-dev \
    libmariadbclient-dev \
    libjpeg62-turbo-dev \
    zlib1g-dev \
    libwebp-dev \
 && rm -rf /var/lib/apt/lists/*

RUN addgroup --system django \
    && adduser --system --ingroup django django

# Requirements are installed here to ensure they will be cached.
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt

# Copy project code
COPY . .
COPY --from=frontend-builder /app/frontend/build /app/frontend/build

RUN python manage.py collectstatic --noinput --clear

# Run as non-root user
RUN chown -R django:django /app
USER django

# Run application
CMD gunicorn django_heroku.wsgi:application

For frontend, what we want is the files in frontned/build, not packages in the node_modules,
For the first build stage, we assign it a name frontend-builder
In the first build stage, we install frontend dependency packages and use npm run build to build static assets, after this command is finished, the built assets would be available in /app/frontend/build
In the second build stage, after COPY . ., we copy /app/frontend/build from the first stage and put it at /app/frontend/build
--from has the value of the build stage, here the value is frontend-builder, which we set in FROM node:12-stretch-slim as frontend-builder
Please note that WORKDIR docker instruction, it sets the working directory for docker instructions (RUN, COPY, etc.) It is very like the cd command in shell, but you should not use cd in Dockerfile.
Since you already know what is docker run and docker build, I want to say nearly all docker instructions above would be executed in docker build. The only exception is the last CMD, it would be excuted in docker run.
We use ENV to set the default env variable.
We added a django user and used it to run the command for security.

Add gunicorn to requirements.txt

gunicorn==20.1.0

Serving static assets

Django only serve media files and static assets in dev mode, so I will show you how to serve them on Heroku in production mode.

To serve static assets, we need to use a 3-party package. whitenoise.

Add whitenoise to requirements.txt

whitenoise==5.3.0

Update django_heroku/settings.py

import os                                                                            # new

MIDDLEWARE = [
   'django.middleware.security.SecurityMiddleware',
   'whitenoise.middleware.WhiteNoiseMiddleware',                                     # new
   # ...
]

STATIC_URL = 'static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')                                       # new

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'frontend/build'),                                        # you can change it
]

STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"      # new

That is all the config you need.

After python manage.py collectstatic --noinput --clear in Dockerfile executed in docker build, all static assets would be put to static directory.
After docker run excecuted on Heroku, Django can serve static assets without issues.

Build Docker image

After you create Dockerfile and put it at root of your Django project, it is recommended to test it in local, this can save you time if something is wrong.

PS: If you have not installed Docker, please check this Docker install Doc

$ docker build -t django_heroku:latest .

In some cases, you might also want to not use the build cache

You can use --no-cache=true option and check leverage-build-cache to learn more.

Docker run

If the docker image has been built without error, here we can keep checking in local env

$ docker run -d --name django-heroku-example -e "PORT=9000" -e "DJANGO_DEBUG=0" -e "DJANGO_ALLOWED_HOSTS=*" -p 9000:9000 django_heroku:latest

# Now visits http://127.0.0.1:9000/admin/

In Dockerfile, we use ENV to set default env variable in docker run, but we can still use -e "PORT=9000" to overwrite the env in run command.
-p 9000:9000 let us can visit the 9000 port in container through 9000 in host machine.
Let's do some check, and below scripts can also help you troubleshoot.

$ docker exec -it django-heroku-example bash

django@709bf089caf0:/app$ ./manage.py shell
Python 3.10.1 (main, Dec 21 2021, 09:50:13) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.conf import settings
>>> settings.DEBUG
False

# cleanup
$ docker stop django-heroku-example
$ docker rm django-heroku-example

Serving media files on Heroku

Since all changes to docker container would be lost during redeploy. So we need to store our media files to some other place instead of Heroku Docker container.

The popular solution is to use Amazon S3 storage, because the service is very stable and easy to use.

If you have no Amazon service account, please go to Amazon S3 and click the Get started with Amazon S3 to signup.
Login AWS Management Console
In the top right, click your company name and then click My Security Credentials
Click the Access Keys section
Create New Access Key, please copy the AMAZON_S3_KEY and AMAZON_S3_SECRET to notebook.

Next, we start to create Amazon bucket on S3 Management Console, please copy Bucket name to notebook.

Bucket in Amazon S3 is like top-level container, every site should have its own bucket, and the bucket name are unique across all Amazon s3, and the url of the media files have domain like {bucket_name}.s3.amazonaws.com.

Now let's config Django project to let it use Amazon s3 on Heroku.

update requirements.txt.

boto3==1.16.56
django-storages==1.11.1

Add storages to INSTALLED_APPS in django_heroku/settings.py

And then update django_heroku/settings.py

MEDIA_ROOT = os.path.join(BASE_DIR, 'media')                                      # new
MEDIA_URL = '/media/'                                                             # new

if 'AWS_STORAGE_BUCKET_NAME' in env:                                              # new
    AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
    AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
    AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
    AWS_S3_REGION_NAME = env('AWS_S3_REGION_NAME')
    AWS_DEFAULT_ACL = 'public-read'
    AWS_S3_FILE_OVERWRITE = False

    MEDIA_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

To secure your Django project, please set AWS_STORAGE_BUCKET_NAME, AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY and AWS_S3_REGION_NAME in ENV instead of project source code (I will show you how to do it in Heroku in a bit)
AWS_S3_FILE_OVERWRITE please set it to False, so this can let the storage handle duplicate filenames problem. (I do not understand why so many blog posts did not mention this)

Please note the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY is for AWS admin, for better security, you should create AWS IAM user and then grant S3 permissions. You can check Bucket owner granting its users bucket permissions to learn more.

Remote DB support

Heroku support many databases, you can choose what you like and add it to your Heroku instance. (I recommend PostgreSQL)

In Heroku, the DB connection string is attached as ENV variable. So we can config our settings in this way.

Update django_heroku/settings.py

if "DATABASE_URL" in env:
    DATABASES['default'] = env.db('DATABASE_URL')
    DATABASES["default"]["ATOMIC_REQUESTS"] = True

Here env.db would convert DATABASE_URL to Django db connection dict for us.

Do not forget to add psycopg2-binary to requirements.txt.

psycopg2-binary==2.9.2

heroku.yml

heroku.yml is a manifest you can use to define your Heroku app.

Please create a file at the root of the directory

build:
  docker:
    web: Dockerfile
release:
  image: web
  command:
    - django-admin migrate --noinput

As you can see, in build stage, docker would build web image from the Dockerfile.
In release stage, migrate command would run to help us sync our database.

From Heroku doc

If you would like to see streaming logs as release phase executes, your Docker image is required to have curl. If your Docker image does not include curl, release phase logs will only be available in your application logs.

We already installed curl in our Dockerfile

Deploy the Django project to Heroku

Now, let's start deploy our Django project to Heorku.

First, we go to Heroku website to login and create an Heroku app. (django-heroku-docker in this case)

After we create the app, we can get the shell command which can help us deploy the project to Heroku.

Then we start to config and deploy in terminal.

$ heroku login
$ heroku git:remote -a django-heroku-docker
$ heroku stack:set container -a django-heroku-docker

# git add files and commit, you can change to main branch
$ git push heroku master:master

heroku stack:set container is important here because it would tell Heroku to use container instead of buildpacks to deploy the project.
You can find the domain of your Heroku app in settings tab. (Heroku has free plan so you can test and learn as you like)

After the code is deployed to Heroku, do not forget to add ENV DJANGO_ALLOWED_HOSTS, DJANGO_DEBUG and DJANGO_SECRET_KEY to Heroku app.

Then check https://django-heroku-docker.herokuapp.com/admin/ to see if everything works.

Add DB add-on

Now you can add db add-on to your Heroku instance, so data of your Django project would be persistent.

Go to the overview tab of your Heroku project, click Configure Add-ons
Search Heroku Postgres and click Provision button.
Now go to settings tab of your Heroku project, click the Reveal Config Vars
You will see DATABASE_URL, it is ENV variable of your Heroku app

# let's check
$ heroku config -a django-heroku-docker

DATABASE_URL:         postgres://xxxxxxx

Do not forget to check Heroku Release log to see if the migration has been run:

Running migrations:
  No migrations to apply.

Add AWS ENV

Now you can add Env AWS_STORAGE_BUCKET_NAME, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_S3_REGION_NAME in Heroku, and check the media uploading feature.

Troubleshoot

If you meet problem, you can use Heroku log to check more details.

Conclusion

In this Django Heorku tutorial, I talked about how to deploy Django project to Heroku using Docker.

You can find the source code django-heroku. I would appreciate that if you could give it a star.

Django Heroku Tutorial Series:

Heroku vs AWS Which is Best for Your Django project

How to deploy Django project to Heroku using Docker

How to deploy Python project to Heroku in Gitlab CI

How to use Heroku Pipeline

Heroku Logs Tutorial

How to monitor Heroku Postgres using heroku-pg-extras

Django Dokku Tutorial Series:

How to deploy Django project to Dokku

How to deploy Django project to Dokku with Docker