personal web log written by izabeera and dryobates

reload queries django

Queries run on Django project’s startup

by dryobates

If you have ever been kicked by sql queries run at the time of loading admin.py, models.py or urls.py modules then you surely want to secure against it in future.

Django load all applications and their “models.py” at startup and executes its code. Then at the first request it loads module defined in settings.ROOT_URLCONF and if you use admin application it loads “admin.py” contained in applications. If you have some code directly in those modules or in definitions of classes (which are executed at module’s load time) then all that code tries to query database. Querying database at startup generally isn’t good idea. It means that no part of Django would be work if database is absent for some reason. Running a lot of queries, especially heave ones, at the time of first request also doesn’t look interesting.

From Django shell console if we would like to check queries run at module startup we could do (needs to be run with DEBUG=True):

$python ./manage.py shell
Python 2.7.8 (default, Sep 17 2014, 04:50:25)
[GCC 4.2.1 Compatible FreeBSD Clang 3.3 (tags/RELEASE_33/final 183502)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.db import reset_queries, connections
>>> import example_app.models
>>> reset_queries()
>>> reload(example_app.models)
<module 'example_app.models' from '/usr/home/jstolarski/projects/www/check_load/example_app/models.pyc'>
>>> connections['default'].queries
[{u'time': u'0.001', u'sql': u'QUERY = u\'SELECT "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined" FROM "auth_user"\' - PARAMS = ()'}]

Why do we reload module? If module was loaded earlier then python does not execute its code again and only adds module to loader module’s global namespace. We reset recorded queries and force module to reload. Only after that operation we can check recorded queries.

Of course checking manually whole project with this method is too tedious task for lazy programmers :) I’ve written management command that checks all applications in project (or only selected one’s if asked):

$ python ./manage.py check_startup_queries | column -t -s,
Module              Queries num  Total time
example_app.models  1            0.0
example_app.admin   7001         0.03300000000000002

Above management command is part of “django-kidocare” [1]

With Django 1.7 system check framework it’s tempting to use it for check that we have done here with command. One caveat is that checks are run before commands like “runserver” and reloading admin modules can lead to some unexpected behaviour like registering admin more then one time. Beside not all queries run at startup are mistakes. Sometimes it’s expected behaviour.

[1]django-kidocare https://bitbucket.org/kidosoft/django-kidocare
dryobates
dryobates
Jakub Stolarski. Software engineer. I work professionally as programmer since 2005. Speeding up software development with Test Driven Development, task automation and optimization for performance are things that focus my mind from my early career up to now. If you ask me for my religion: Python, Vim and FreeBSD are my trinity ;) Email: jakub@stolarscy.com

Archive

Tag cloud