{"id":1882,"date":"2021-11-04T13:51:59","date_gmt":"2021-11-04T11:51:59","guid":{"rendered":"http:\/\/52.91.248.125\/speeding-up-your-python-django-test-suite\/"},"modified":"2023-02-28T11:52:10","modified_gmt":"2023-02-28T09:52:10","slug":"speeding-up-your-python-django-test-suite","status":"publish","type":"post","link":"https:\/\/www.orfium.com\/engineering\/speeding-up-your-python-django-test-suite\/","title":{"rendered":"\u23f1\ufe0f Speeding up your Python & Django test suite"},"content":{"rendered":"
\n
\"\"
Artwork by @vitzi_art<\/a><\/figcaption><\/figure><\/div>\n\n\n

Less time waiting, more time hacking!<\/h3>\n\n\n\n

Yes yes, we all know. Writing tests and thoroughly running them on our code is important. None of us enjoy doing it but we almost all see the benefits of this process. But what isn\u2019t as great about testing is the waiting, the context shifting and the loss of focus. At least for me, this distraction is a real drag, especially when I have to run a full test suite.<\/p>\n\n\n\n

This is why I find it crucial to have a fine-tuned test suite that runs as fast as possible, and why I always put some effort into speeding up my test runs, both locally and in the CI. While working on different  Python \/ Django projects I\u2019ve discovered some tips & tricks that can make your life easier. Plenty of them are included in various documentations, like the almighty Django docs<\/a>, but I think there\u2019s some value in collecting them all in a single place.<\/p>\n\n\n\n

As a bonus, I\u2019ll be sharing some examples \/ tips for enhancing your test runs when using Github Actions, as well as a case study to showcase the benefit of all these suggestions.<\/p>\n\n\n\n

The quick wins<\/h3>\n\n\n\n

Running a part of the test suite<\/h4>\n\n\n\n

This first one is kind of obvious, but you don\u2019t have to run the whole test suite every single time. You can run tests in a single package, module, class, or even function by using the path on the test command.<\/p>\n\n\n\n

> python manage.py test package.module.class.function\nSystem check identified no issues (0 silenced).\n..\n----------------------------------------------------------------------\nRan 2 tests in 6.570s\n\nOK<\/code><\/pre>\n\n\n\n
Keeping the database between test runs<\/h4>\n\n\n\n
By default, Django creates a test database for each test run, which is destroyed at the end. This is a rather slow process, especially if you want to run just a few tests! The --keepdb<\/kbd><\/em> option<\/a> will not destroy and recreate the database locally on every run. This gives a huge speedup when running tests locally and is a pretty safe option to use in general.<\/p>\n\n\n\n
> python manage.py test <path or nothing> --keepdb\nUsing existing test database for alias 'default'...          <--- Reused!\nSystem check identified no issues (0 silenced).\n..\n----------------------------------------------------------------------\nRan 2 tests in 6.570s\n\nOK\nPreserving test database for alias 'default'...              <--- Not destroyed!<\/code><\/pre>\n\n\n\n
This is not as error-prone as it sounds, since every test usually takes care of restoring the state of the database, either by rolling back transactions or truncating tables. We\u2019ll talk more about this later on.<\/p>\n\n\n\n
If you see errors that may be related with the database not being recreated at the start of the test (like IntegrityError<\/kbd><\/em>, etc), you can always remove the flag on the next run. This will destroy the database and recreate it.<\/p>\n\n\n\n
Running tests in parallel<\/h4>\n\n\n\n
By default, Django runs tests sequentially. However, whether you\u2019re running tests locally or in your CI (Github Actions, Jenkins CI, etc) more often than not you\u2019ll have multiple cores. To leverage them, you can use the --parallel<\/kbd><\/em> flag<\/a>. Django will create additional processes to run your tests and additional databases to run them against.<\/p>\n\n\n\n
You will see something like this:<\/p>\n\n\n\n
> python3 manage.py test --parallel --keepdb\nUsing existing test database for alias 'default'...\nUsing existing clone for alias 'default'...        --\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |    => 12 processes!\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...         |\nUsing existing clone for alias 'default'...        -- \n\n< running tests > \n\nPreserving test database for alias 'default'...\n... x10 ...\nPreserving test database for alias 'default'...<\/code><\/pre>\n\n\n\n
In Github runners, it usually spawns 2-3 processes.<\/p>\n\n\n\n
When running with --parallel<\/kbd>, Django will try to pickle tracebacks from errors to display them in the end. You\u2019ll have to add tblib<\/kbd> as a dependency to make this work.<\/em><\/p><\/blockquote>\n\n\n\n
Caching your Python environment (CI)<\/h4>\n\n\n\n
Usually when running tests in CI\/CD environments a step of the process is building the Python environment (creating a virtualenv, installing dependencies etc). A common practice to speed things up here is to cache this environment and keep it between builds since it doesn\u2019t change often. Keep in mind, you\u2019ll need to invalidate this whenever your requirements change.<\/p>\n\n\n\n
An example for Github Actions could be adding something like this to your workflow\u2019s yaml file:<\/p>\n\n\n\n
- name: Cache pip\n  uses: actions\/cache@v2\n  with:\n    # This path is specific to Ubuntu\n    path: ${{ env.pythonLocation }}\n    # Look to see if there is a cache hit for the corresponding requirements file\n    key: ${{ env.pythonLocation }}-${{ hashFiles('requirements.txt','test_requirements.txt') }}<\/code><\/pre>\n\n\n\n
Make sure to include all your requirements files in hashFiles<\/kbd><\/em>.<\/p><\/blockquote>\n\n\n\n
If you search online you\u2019ll find various guides, including the official Github <\/em>guide<\/em><\/a>, advising to cache just the retrieved packages (the pip cache). I prefer the above method which caches the built packages since I saw no speedup with the suggestions there.<\/em><\/p><\/blockquote>\n\n\n\n
The slow but powerful<\/h3>\n\n\n\n
Prefer TestCase<\/kbd> instead of TransactionTestCase<\/kbd><\/h4>\n\n\n\n
Django offers 2 different base classes for test cases: TestCase<\/kbd><\/em> and TransactionTestCase<\/kbd><\/em>. Actually it offers more, but for this case we only care about those two.<\/p>\n\n\n\n
But what\u2019s the difference? Quoting the docs<\/a>:<\/p>\n\n\n\n
Django\u2019s TestCase<\/kbd><\/em> class is a more commonly used subclass of TransactionTestCase<\/kbd><\/em> that makes use of database transaction facilities to speed up the process of resetting the database to a known state at the beginning of each test. A consequence of this, however, is that some database behaviors cannot be tested within a Django TestCase<\/kbd><\/em> class. For instance, you cannot test that a block of code is executing within a transaction, as is required when using select_for_update()<\/kbd>. In those cases, you should use TransactionTestCase<\/kbd><\/em>.<\/p>\n\n\n\n
TransactionTestCase<\/kbd> and TestCase<\/kbd> are identical except for the manner in which the database is reset to a known state and the ability for test code to test the effects of commit and rollback:<\/em><\/p>\n\n\n\n