Speeding up migrations when using Django Tenants, in prod and in tests

Published: 2026-09-14

Django Tenants and Migrations

Django Tenants is a great library for building SaaS applications. It works by separating tenant data into PostgreSQL schemas. PostgreSQL schemas are a bit like a namespace in the database, so the library creates a namespace for each tenant. The schemas behave almost like separate databases (with some exceptions), which allows you to put the same table in each schema and query just the one you need to serve a specific customer. If you are interested in more details, I had a talk at EuroPython about this approach and its pros and cons.

When you use Django Tenants, you have a shared schema (often called the public schema) which holds the information about the tenants and any other shared data, and then a schema for each tenant. All Django applications are split into two categories - shared and tenant-specific. When running migrations, only the relevant migrations run in each schema - but importantly, any tenant migrations need to run in every schema. So if you have many tenants, you need to run migrations in a lot of schemas.

This is a major con of the Django Tenants approach, so most people actually recommend not using Django Tenants when expecting a high number of tenants - managing a large number of schemas causes quite significant overhead.

The slowness of migrations presents itself in quite a few places once your application grows.

Adding a new tenant - migrations need to run from all apps on the new schema.
Deploying any migrations to production - they need to run on all tenants (and the public schema).
Running tests - the migrations need to run on the public schema and any tenants you create in tests.

We were increasingly running into these issues at Xelix where we have been using Django Tenants since 2020. The issues came from two sides: we have been adding a lot of new customers to the platform (always a good sign), but the platform has also grown in complexity and therefore the number of models has significantly increased.

That is why I wrote django-tenants-smart-executor library which contains two features which make running migrations faster. In the second half of this blog, I will also discuss how to optimise your tests when using Django Tenants (the last one will also be relevant to all Django users)!

Django Tenants Smart Executor

You can find the library on PyPI and the source code on GitHub. Both have READMEs with instructions on installation and usage.

Trick 1: Not running migrations if there's nothing to apply 🔗

It's quite common to always run migrations on deployment and let Django (or Django Tenants) deal with detecting whether anything needs migrating. However, with Django Tenants, even if all schemas are fully up to date, this can be quite slow.

Exactly how slow obviously depends on the hardware, but most importantly on the complexity of the application. This is because Django Tenants actually just runs the migrate command in each schema under the hood when running python manage.py migrate_schemas, and the default Django migrate command does more than run migrations. It also runs all regular Django checks and tries to detect any changes to models, so it can show the helpful message about unreflected changes. In local development those two things are quite useful, but when you are deploying to production, you are probably pretty sure that correct migrations are present (it's pretty easy to detect in CI).

What it does instead is make the migrations quite slow, because those checks need to run in every schema. This leads to a sad situation of deploying to production, and even if there are no migrations to run, you will be waiting for nothing to happen for ages. At Xelix, this was about 5 seconds per tenant, which is fine locally if you only have a couple tenants, but in production we have several orders of magnitude more, slowing down the entire release cycle.

Turns out, however, it's actually quite easy to detect if you have migrations to run! And Django Tenants is a great modular library, and it actually exposes a way for you to redefine how migrations are run, using the GET_EXECUTOR_FUNCTION Django setting.

This is where the library finally comes in. It provides alternative executors for Django tenants which before running anything detect if there are any migrations which need applying. If yes, they will run everything just like normal. If they detect that there are no migrations to apply, it will short-circuit and skip all the checks and migration detection. The schema_migrated signal which is normally issued by Django Tenants on migration will still be triggered.

To start using it, all you need to do is install the library and configure the GET_EXECUTOR_FUNCTION setting in your settings.py:

GET_EXECUTOR_FUNCTION = "django_tenants_smart_executor.load_executor"

It works by hooking into the migration recorder that Django uses to store which migrations have been applied and compares them to migrations in your codebase. It works for squashed migrations as well! The standard and multiprocessing executors are supported; however, the subprocess one added in Django Tenants 3.12 is not yet supported.

Trick 2: Skipping apps not relevant for the schema 🔗

There are two aspects to Django Migrations, both serving their own function. The first is the database operations, the part which actually makes changes to the database, for example, adding a new column to a table. The second is the state operations, which do not make any changes to the database, but instead are used to track what the models looked like at the time of the migration. This state is used to detect what changes need to be made to get the database to match your current models.

Django migrations do not have access to your git history, so it can't exactly know how the model looked before you ran python manage.py makemigrations, so it rebuilds how the previous version of the model worked from these state operations. For example, if you add a new field to your model, Django needs to go through every past migration in memory to know that it needs to add an AddField operation to the new migration. In general, the state part of the migration is primarily for detecting migration changes, not for executing them.

One would hope that running some memory operations on a few migrations would be quite fast? Well, it really depends on how complex your data schema is, how many tables, how many foreign keys you have; and that is multiplied by how many migrations you have in the history. The more you have of these, the slower it will actually be. Others have raised that migrations can be quite slow (and memory-hungry), precisely because of the state migrations. You can find much discussion about this on the Django forum, for example, here, here and here.

This will be an issue even if you don't use Django Tenants, but Django Tenants makes this issue much worse. Either way, you want to periodically squash migrations. I recommend django-remake-migrations!

The way Django Tenants handles migrations is that it overrides the allow_migrate configuration and returns True depending on what migration is running within which schema. This is great, it does what it needs to, but Django actually always runs the state migrations. So, for example, when running migrations in the public schema, you will need to wait for the schema operations for all the tenant schema apps, and vice versa.

The extra time you will spend on migrations is determined by the make-up of your apps. If you have more migrations in the public schema, you will wait longer migrating tenants; if you have more migrations in the tenant schema, you will wait longer migrating public schema. Either way, if you consider migrating the public schema and one tenant, you are wasting half of the time applying state migrations which don't matter.

This is the second big issue the library solves; by skipping the irrelevant migrations based on the schema. To start using it, it's slightly more effort than the first trick. GET_EXECUTOR_FUNCTION needs to be configured to the library's version, otherwise it's not going to work. Then, you will need to replace every single django.db.migrations import with django_tenants_smart_executor.migrations in your migration files. All public classes and functions from the module are re-exported, so you can do a simple find & replace. The ruff rule TID251 is useful for enforcing this.

[ruff.lint.flake8-tidy-imports.banned-api]
"django.db.migrations" = { msg = "Use django_tenants_smart_executor.migrations" }

Finally, you need to configure your isolation scope with the SMART_EXECUTOR_LIMIT_STATE_TO_SCHEMA Django setting. It's possible you can't ignore all state migrations, because if you have a dependency between the public and tenant schema, the migration state would not know the models from the opposite side. For example, if your user table is in the public schema, and you have a foreign key to the user from the tenant schema, when applying the migration in tenant schema, the migration framework needs to know about it, and we need to run the state migrations of the user app.

Full (full). You can use this if there are no relationships between the public and tenant schemas.
Public (public). This will skip the state migrations of tenant apps when running on public schema. This is safe to use when there are no relationships from the tenant schema to the public schema.
Tenant (tenant). This will skip the state migrations of public apps when running on tenant schema. This is safe to use when there are no relationships from the public schema to the tenant schema.

On top of this, you can further configure exceptions with SMART_EXECUTOR_LIMIT_STATE_EXCEPTIONS_MAP. That option should be a dictionary of a booleans (whether it's in the tenant schema) to iterables of apps which should be migrated regardless of the mode. For example, with the following settings, the only public app that will have state migrations applied in the tenant schemas will be account. There will be no state migrations for tenant apps performed in public schema.

SMART_EXECUTOR_LIMIT_STATE_TO_SCHEMA = "full"
SMART_EXECUTOR_LIMIT_STATE_EXCEPTIONS_MAP: dict[bool, set[str]] = {
    True: {  # in tenant schema, do migrate these public apps
        "account",
    },
    False: set(),  # in public schema, there are no exceptions for any tenant apps
}

Under the hood, this is implemented by overriding the state migration of each migration operation to only run it based on the configured setting, essentially replicating the allow_migrate option.

Optimise your tests!

The first two tricks primarily help to reduce migration time in production and on your local, but also have benefits in tests, mainly the second one. At Xelix, implementing that one halved the setup time for our tests!

The fourth and fifth tricks described in the section are relevant for all Django projects, not just Django Tenants; however, they are only relevant if you run migrations within tests. If you fake migrations with --run-syncdb, this whole section is irrelevant.

Trick 3: Generate one tenant, duplicate schema to other tenants 🔗

Running tests, you need to create the public schema, and you probably need to create at least one tenant. So you can't run away from the cost of running the migrations on the public schema and one tenant schema. If your tests, however, need more than one tenant to exist in the database for you to run tests, you want to avoid running migrations on both the schemas. Assuming you are not using multiple tenant types, you can duplicate the schemas from one schema to another, saving yourself from running the migrations twice (or even more, the more tenants you have).

Honestly, duplicating schemas is not the easiest thing to do, but it is really worth it. Django Tenants has a management command for this called clone_tenant, but for some reason I no longer remember, it doesn't work for all schemas; I think for us at Xelix it was due to materialised views? It uses pg-clone-schema under the hood. Your mileage may vary.

There is a dumb but fairly functional way to replicate schemas using pg_dump. The script goes something like this, you will obviously need to replace the names of databases, schemas, etc.; experiment a little, but this should get you quite far.

# generate schema dumps for public schema and the default test schema from 'test_database'
pg_dump --inserts --host=$DB_HOST -U $DB_USER --schema test_schema test_database > test_schema.sql
pg_dump --inserts --host=$DB_HOST -U $DB_USER --schema public test_database > public.sql

# create a second helper database
psql --host=$DB_HOST -U $DB_USER -bq -c "DROP DATABASE IF EXISTS test_database_schemas"
psql --host=$DB_HOST -U $DB_USER -bq -c "CREATE DATABASE test_database_schemas"

# load public schema and test_schema
psql --host=$DB_HOST -U $DB_USER -bq -d test_database_schemas -f public.sql
psql --host=$DB_HOST -U $DB_USER -bq -d test_database_schemas -f test_schema.sql

# rename test_schema to other_test_schema; dump to a file
psql --host=$DB_HOST -U $DB_USER -bq test_database_schemas -c "ALTER SCHEMA test_schema RENAME TO other_test_schema"
pg_dump --inserts --host=$DB_HOST -U $DB_USER --schema other_test_schema test_database_schemas > other_test_schema.sql

# load back to test_database
psql --host=$DB_HOST -U $DB_USER -bq -d test_database -f other_test_schema.sql

Trick 4: Generate one database, replicate to multiple workers 🔗

If you are using pytest-xdist, your migrations need to run in each worker (each gets its own database when using pytest-django) - thankfully, they do run in parallel, but this can still cause some pain. One pain is possibly overwhelming the PostgreSQL server by N parallel processes running a lot of database migrations. The second pain is cost: you might be running your tests on a fairly powerful server with a lot of CPUs, so you get your test results pretty quickly, but then at the start of the tests you are waiting for the migrations to run in all the N parallel processes, before the tests themselves actually run. Even if it's a couple of minutes, a couple of minutes of a powerful server adds up across hundreds of CI runs in pull requests, merge queue, and main.

To fix the first pain of overwhelming the database server, you could run the test migrations just for one database and then replicate that database for all the other workers. The following bash snippet replicates test_database into $N databases, which can be picked up by pytest-xdist if using --reuse-db. This can be nicely combined with the previous trick of duplicating schemas.

for i in `seq 0 $N`; do
  psql --host=$DB_HOST -U $DB_USER -c "CREATE DATABASE test_database_gw$i WITH TEMPLATE test_database;" &
done

To fix the cost problem, you can run this initial migration on a less powerful runner than your final test runner, saving on dead CPU time. The test setup will not be materially quicker by replicating the databases, but you can save some good money doing it that way.

Trick 5: Cache your migrations between CI runs 🔗

What can materially speed up your test setup is the final trick I will talk about here, which concerns caching your migrated database for test purposes. If you are running CI, and no migrations have changed since the last commit, there is no point setting up the database from scratch; you can reuse the initial DB state from the previous CI. This speeds up the CI in all cases where no migrations have changed, improving the developer experience. At Xelix we found only about 15% of pull requests touched migrations, so the speedup can be present on a good proportion of PRs.

As per usual, one of the hard parts is defining the cache key; I recommend you include every single migration file, all Django settings, and the CI file as well for good measure.

The overall process then looks something like this:

Boot a cheap runner
Check if a database dump exists for the current cache key
If it does, restore the dump from the cache
If it doesn't, run the migrations, replicate schemas if you need to, and create a dump
Store the dump to cache for the current cache key
Create an artefact with the dump file
Boot the expensive runner where your tests will run
Restore the artefact and load it into the database
Replicate the database into N databases you will need for pytest-xdist
???
Profit!

Summary

So, hopefully, now you have a better understanding of how to mitigate some of the costs of using Django Tenants. They are real costs, but Django Tenants has real benefits, so they are worth it, and there are mitigations. I hope you have also learned something about migrations along the way!

Get in touch with me at my email mikulaspoul [at] gmail [dot] com if you have any comments, questions, corrections, or suggestions!