Does order matter in urls.py?

MP 58: Yes, but not for the reason I originally thought.

Pattern matching is an interesting problem in programming. It applies any time you have data coming in that should fit some form of consistent pattern, and you need to match each piece of data to a particular action.

In Django, every request that arrives at the server needs to be matched to a specific view. This is a classic example of pattern matching; Django has to match a URL pattern with a function or class that will process the request and return an appropriate response.

Someone asked recently if the order of URL patterns in a Django project matters. I know the order is significant, but I wasn’t quite sure how to explain why the order matters. Digging into this question was a lot more interesting than I anticipated.

The original question

Here are the URL patterns in the root urls.py for the Learning Log project in Python Crash Course:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('accounts/', include('accounts.urls')),
    path('', include('learning_logs.urls')),
]

A reader asked if the order of the patterns listed here matters. Specifically, they wanted to know if the line that includes the URLs for the accounts app can be placed last in this list of patterns.

This question came up because the project is developed in stages, and the root urls.py looks like this a little earlier in the project:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('learning_logs.urls')),
]

They were basically asking if we could just add the new pattern for the accounts app on the last line, like this:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('learning_logs.urls')),
    path('accounts/', include('accounts.urls')),
]

In the Learning Log project, you can shift these lines without impacting URL routing for the project. However, the URL pattern that starts with an empty string should be placed last in a Django project. To understand why, let’s look at some specific URLs in a smaller example project.

A piano store

Imagine you’re building a web site for a piano store. At first, the piano store only sells pianos. So your project starts out with a single app called pianos.

Here’s urlpatterns in the root urls.py file:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('pianos.urls')),
]

This file has two patterns: the URLs for Django’s default admin app, and the URLs associated with the pianos app. The pattern for the pianos app matches an empty string, which will capture any URL that doesn’t start with admin/:

https://my-piano-store.com
https://my-piano-store.com/about
https://my-piano-store.com/contact
https://my-piano-store.com/faq

In this simple version of the project, there should be no conflict between the URLs for the admin app and the pianos app.

Now we sell synthesizers!

The piano store is doing well, and has quite a lot of acoustic and digital pianos for sale. The physical store is large, and the online store has even more options available. It’s doing so well, they open a small side room with a few synthesizers.

The developer decides to make a new app called synths, to keep the synthesizers separate from the pianos in the project. Here’s the updated root urlpatterns:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('pianos.urls')),
    path('synths/', include('synths.urls')),
]

The developer added the new pattern to the end of the list, because they’re used to adding new items at the last position in a list if it already exists.

Does this order work? Or do you have to place the URL pattern that matches the empty string last in a list of URL patterns?

Let’s make some pages, and find out.

The app URL patterns

To experiment with this, we need to set up a few pages that we can work with. Here’s some URL patterns for the pianos app:

# pianos/urls.py

app_name = 'pianos'
urlpatterns = [
    path('', views.index, name='index'),
    path('about/', views.about, name='about'),
    path('contact/', views.contact, name='contact'),
    path('faq/', views.faq, name='faq'),
]

These patterns match the four URLs listed earlier: a home page, an about page, a contact page, and a faq page.

To keep things simple, we won’t deal with templates at all. We’ll just use HttpResponse in views.py to return simple responses, making it clear which app is handling any given URL:

# pianos/views.py

from django.http import HttpResponse

def index(request):
    msg = "Welcome to the Piano Store!"
    return HttpResponse(msg)

def about(request):
    msg = "About the Piano Store"
    return HttpResponse(msg)

...

Each view defines a short message, and returns that message as an HTTP response.

The home page for the demo Piano Store.

Here’s the URL patterns for synths:

# synths/urls.py

app_name = 'synths'
urlpatterns = [
    path('about/', views.about, name='about'),
    path('faq/', views.faq, name='faq'),
]

There are two pages in the section of the site dedicated to synthesizers.

URL routing refresher

If you haven’t already done so, I highly recommend spending some time reading the documentation for Django’s URL dispatcher. If the word dispatcher is unclear, it’s the part of Django that matches incoming URLs to a view in the project. Let’s revisit how it works, by focusing on how a specific URL in the piano store project is handled.

Let’s use a synths URL, because that’s actually one of the more interesting URLs to reason about in the current version of the project. For this part of the discussion, we’ll use a URL that resolves in a local instance of the project, using runserver:

http://localhost:8000/synths/about

First, the URL dispatcher ignores the protocol (http://) and the domain (localhost:8000/). It only has to process the following string:

synths/about

Django first looks in the root urls.py:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('pianos.urls')),
    path('synths/', include('synths.urls')),
]

It goes through each of these patterns in order, until it finds a match. The first pattern, for URLs starting with admin/, doesn’t match synths/about.

The next pattern it looks at is the empty string. This pattern actually matches every URL that comes in. This is a really important point: every URL will match the empty string, so every incoming URL will be checked against all the URLs defined in an app that matches the empty string. In this example, Django looks through all the patterns in pianos/urls.py to see if any match the string synths/about:

# pianos/urls.py

urlpatterns = [
    path('', views.index, name='index'),
    path('about/', views.about, name='about'),
    path('contact/', views.contact, name='contact'),  
    path('faq/', views.faq, name='faq'),
]

None of these patterns start with synths/, so none of them match the requested URL.

Django moves on to the next pattern in the root urls.py, which matches the string synths/. The URL dispatcher has matched the first part of the string it’s working with, so now all it has left to match is:

about

The match for synths/ sent the dipatcher to synths.urls, so it looks through those patterns to see if any match the string about:

# synths/urls.py

urlpatterns = [
    path('about/', views.about, name='about'),
    path('faq/', views.faq, name='faq'),
]

The first pattern listed here matches, so the URL dispatcher hands execution off to the about() function in synths/views.py. Here’s the resulting page:

The about page for the synths app.

Verifying Django’s behavior

Another section of the Django docs that’s very much worth reading is the Request and response objects page. Every view function receives a request object, and the more you know about this object the more you can do with it.

One interesting attribute of the request object is resolver_match. This object contains information about the URL, after it’s been matched to a view. The information we’re most interested in is request.resolver_match.tried; this is a list of the URL patterns that were tried before finding a match.

The URL we just examined matched the about page in the synths app. So we can work with the request object in that view function, and see the patterns that were tried:

# synths/views.py

def about(request):
    for pattern in request.resolver_match.tried:
        print(pattern)

    msg = "About our synthesizers"
    return HttpResponse(msg)

We loop over the list of patterns that were tried, and print each pattern. Now we can reload the URL http://localhost/synths/about, and in the terminal where runserver is running we’ll see all the URL patterns that were tried before a match was found:

(admin:admin) 'admin/'

'pianos.urls' <URLPattern '' [name='index']>
'pianos.urls' <URLPattern 'about/' [name='about']>
'pianos.urls' <URLPattern 'contact/' [name='contact']>
'pianos.urls' <URLPattern 'faq/' [name='faq']>

'synths.urls' <URLPattern 'about/' [name='about']>

This is a condensed version of the output, but it shows that the analysis above was accurate. Django’s URL dispatcher first tried to match against the admin app, but that didn't work. It then tried all the patterns in pianos, but none of them matched either. Finally, it found a match in the about/ pattern from synths.urls.

This is the same kind of information that’s included in Django’s debug pages. We can see that by intentionally requesting a page that doesn’t exist. Here’s the debug page for the URL http://localhost:8000/synths/nonexistent_page:

The 404 debug page shows all the places the URL dispatcher looked in an attempt to serve the requested URL.

Conflicting URLs

Originally, I thought you had to list a root URL pattern that matches the empty string last, because that pattern would match some URLs that are meant to be routed to apps that are listed later in urlpatterns. But it was harder to come up with an actual example of that happening than I thought.

Let’s revisit the root urls.py, and try to come up with a URL that isn’t handled correctly because of the order of the URL patterns:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('pianos.urls')),
    path('synths/', include('synths.urls')),
]

Can we come up with a URL that should route to a pattern in synths, but accidentally gets matched by the pianos app? Well, every URL that gets matched to the synths app has to start with synths/. So the only way for any of these URLs to get overridden by the pianos app is for the pianos app to match a URL that starts with synths/. Let’s make a scenario where that might happen.

Imagine the online section of the store has really grown, and they sell a lot of new synthesizers online. There are a whole bunch of synthesizers available, all reached through a URL like this:

https://my-piano-store.com/synths/prophet_rev2

In this URL, prophet-rev2 is a model name.

Everything’s working fine, but at some point someone in the piano department takes a synth as a trade-in, and they post it as a special on the pianos section of the store. They do so by adding a new URL to pianos/urls.py:

# pianos/urls.py

urlpatterns = [
    path('', views.index, name='index'),
    path('about/', views.about, name='about'),
    path('contact/', views.contact, name='contact'),
    path('faq/', views.faq, name='faq'),
    path('synths/prophet-rev2/', views.prophet_rev2,
            name='prophet_rev2'), 
]

Here’s the conflict: Because this set of patterns is searched before the URLs for synths, anyone who visits the URL https://my-piano-store.com/synths/prophet-rev2 will see the page for the used instrument, and no one will see the page for the new version of that instrument!

Changing the order doesn’t fix things

I used to think that if this ever happened, changing the order of the root urls.py would fix things. I thought if you put the pattern matching the empty string last, everything would work:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('synths/', include('synths.urls')),
    path('', include('pianos.urls')),
]

This fixes the issue of people looking for information about a new Prophet Rev 2. The URL dispatcher now examines URLs starting with synths/ before it looks at patterns in the pianos app, so the URL https://my-piano-store.com/synths/prophet-rev2 now matches the page for a new instrument in the synths app.

However, this means no one can ever see the page for the used instrument in the pianos app! Every URL gets matched to a single view, and the dispatcher stops looking for a match as soon as it finds one. The first match always wins.

That’s worth repeating, in a slightly different way: every URL gets matched to a single view, even if it matches more than one pattern. A URL that matches more than one pattern will get matched to the first view that comes up in the search order.

A good use of overriding URLs

There’s an important reason for this behavior. It means you can always override a more general pattern with a specific one, by listing the specific pattern earlier in the list of patterns that are searched.

Imagine an instrument called Fire Sounds 8000 has been listed in the synths store. But it’s been found to burst into flames as people are playing, so there’s a highly urgent recall. The piano store wants to make sure everyone looking for this instrument sees the recall notice, and no one sees the regular page for that instrument.

They can list the pattern for that instrument first in the root urls.py:

# urls.py

urlpatterns = [
    path('synths/firesounds8000/', piano_views.recall),
    path('admin/', admin.site.urls),
    path('synths/', include('synths.urls')),
    path('', include('pianos.urls')),
]

Since this is the very first pattern listed in the root urls.py, any URL that matches this pattern will be routed to the recall view. This pattern doesn’t have to be listed first, and it doesn’t have to be listed in the root urls.py. As long as it’s listed before any other patterns that would match this URL, you can send users to the appropriate recall page.

This is a contrived example, but this behavior is used to override pages in many real world projects. For example say you want to use most of the pages in a third party app. However, you want to override a few specific pages. You can do this by adding patterns for those pages in your urls.py file, in any position before the third party app’s URLs are included.

An unlikely issue?

Projects that are organized reasonably are quite unlikely to have conflicts like this. For example most real-world projects would have more structure than what’s shown here.

Consider how much simpler things are if each of the apps has a shop/ section:

# pianos/urls.py

urlpatterns = [
    path('', views.index, name='index'),
    path('about/', views.about, name='about'),
    path('contact/', views.contact, name='contact'),
    path('faq/', views.faq, name='faq'),
    path('shop/', views.shop, name='shop'),
]

# synths/urls.py

urlpatterns = [
    path('about/', views.about, name='about'),
    path('faq/', views.faq, name='fan'),
    path('shop/', views.shop, name='shop'),
]

Now, if someone in the piano department decides to add a synthesizer, they’re likely to post it in their shop. But the pianos/shop/ URLs are all distinct from the synths/shop/ URLs. Here’s where the used synth in the piano shop might be:

https://my-piano-store.com/shop/prophet-rev2

And here’s where the new models in the synth shop might be:

https://my-piano-store.com/synths/shop/prophet-rev2

You have to work kind of hard to come up with a URL conflict in a project that’s reasonably well organized.

Apps are separate, but not completely independent

Django implemented the apps model a long time ago, so that projects could be built from smaller components. This has the benefit of allowing some components to be reusable, which has led to the rich diversity of third-party apps we have today. It also allows parts of a project to be developed and maintained more easily, with some level of isolation.

However, apps do interact within a project. The URL dispatcher is one of the areas where there’s some overlap between apps. Apps can mostly make their own decisions about how they handle things, but they’re not completely isolated from other apps in a project. If a conflict between URLs does arise, a coherent system for structuring URLs must be developed that works for all apps in the project.

Why you should list the empty string pattern last in urls.py

We started with the question of whether order matters in a urls.py file. Order does matter, because Django will pass the current request to whichever URL pattern matches first. Usually, this is a good thing because it lets you override URL patterns that you want to handle in a particular way.

When URLs conflict in an undesirable way, you’ll need to sort out a more specific overall URL scheme. There are many ways to do this, and they depend on the context of the project.

URL conflicts aren’t likely to come up in most of the Django projects you work on. Still, you should list the empty string pattern last, like this:

# urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    path('synths/', include('synths.urls')),
    path('', include('pianos.urls')),
]

There are several reasons to prefer this ordering:

This structure is more efficient. If the pattern that starts with an empty string isn’t listed last, any URL that matches a later pattern will still have to run through all the patterns in the app matching the empty string. That could be a long list, and a lot of effort would be wasted searching through URLs that should never match.

This ordering is much more readable. When you’re considering how a URL would be routed you can strip the protocol and domain, look at what’s left, and see which path the URL will take out of the root urls.py file. If the empty string pattern were listed in the middle, you’d have to consider all the URLs in that app’s urls.py file.

Finally, there’s a general principle in pattern matching that specific patterns should be checked before more general patterns. Following this principle tends to be more efficient, and avoids accidental mismatches.

What about skipping the empty string?

It’s sometimes recommended to not match the empty string to an entire app’s URLs. But there are a lot of projects with one main app, and you don’t necessarily want to clutter most of your project’s URLs with an unneeded app name.

Consider the piano store example one last time. The main about page is about the whole store; it’s not just about pianos. So, we probably want this URL:

https://my-piano-store.com/about

and not:

https://my-piano-store.com/pianos/about

Similarly, we’d rather have:

https://my-piano-store.com/shop

instead of:

https://my-piano-store.com/pianos/shop

If this last example is unclear, consider that the main shop would almost certainly sell more than just pianos. It will likely have things like stools and sheet music for sale as well. So you probably want URLS like:

https://my-piano-store.com/shop/pianos
https://my-piano-store.com/shop/stools
https://my-piano-store.com/shop/sheet_music

rather than:

https://my-piano-store.com/pianos/shop/pianos
https://my-piano-store.com/pianos/shop/stools
https://my-piano-store.com/pianos/shop/sheet_music

Most people don’t look very closely at URLs these days, so this probably wouldn’t cause significant problems. But some people notice and appreciate clean URLs. There are certainly arguments to be made for matching your main app’s URLs to the empty string.

Conclusions

Django makes building web apps easier, but any web framework has lots of parts to it. Often times what seem like simple questions end up getting quite deep when you start pulling back the layers. This is especially true when building working examples, and when you start to think about real-world projects.

Django’s URL dispatcher has a pretty clear resolution order. Keep in mind that it will stop at the first match it finds. In general, you want to list specific patterns before more general ones, and match the empty string last in a list of patterns.

While you might not want to match the empty string to an entire app, you should almost always match the empty string to a view somewhere in your set of URL patterns. Otherwise users will see a 404 error when they issue a request with your bare domain name, such as https://my-piano-store.com.

Django has excellent documentation, so consider reading about its URL dispatcher and the request and response objects. You’ll almost certainly come away with a better understanding of how Django works, and how to structure your own projects a little better. Even if you don’t change the way you build your projects, you’ll have a better rationale for how you are doing things.

Resources

You can find the code files from this post in the mostly_python GitHub repository.

---

Note: I asked about this topic on the Django Forum, and got some really helpful feedback before writing this post. If you work with Django, it’s a great place to get answers to your questions, and to offer help to others as well. Good luck answering before Ken though. :)