As programmers, we choose to spend a significant part of our lives chasing bugs. I think it’s probably fair to say we spend more time staring at code that’s broken in some way, than code that’s working exactly as it’s supposed to.

In this post I’ll share some of the work I’ve been doing to move this newsletter to Ghost, and how it relates to troubleshooting errors in the early stages of a project.

Two kinds of bugs

Broadly speaking, there are two kinds of bugs. One class of bugs causes code to break. At some point while executing the code in a project, Python encounters an error it can’t resolve effectively enough to continue running. This kind of error leads to a traceback, which hopefully has enough information to point you toward the source of the issue.

The second kind of bug is a logical error. In this case, the program is able to complete its execution. If an issue arises during the program’s execution, Python can find a way around it without generating a traceback. These bugs are often harder to sort out. They usually depend on a deeper understanding of the problem you’re trying to solve, and a detailed understanding of how the code addresses that problem.

Back to code blocks

Early last year I wrote a long post about how Substack should improve their code blocks if they want to support good technical writing. They haven’t done anything to address this, and one of the main reasons I’m moving off of Substack is that it’s not designed well for technical writing.

As an example, here’s a block of code I want to use in an upcoming post:

from pathlib import Path

path = Path("coffees.txt")
contents = path.read_text()

print(contents)

This is about as much as you can do with a code block on Substack. There’s no syntax highlighting, and there’s no built-in way to indicate which lines you’re focusing on in the text. Most of us would never put up with this kind of limitation in a text editor or IDE. The lack of modern styling also limits how clearly you can talk about specific lines of code.

With its default settings, Ghost doesn’t do much better in email posts:

There are two things Ghost does better out of the box. First, you can add captions to code blocks, which makes it easier to show which file a listing refers to.

Second, Ghost lets you do more with the web version of each post. For example, you can apply syntax highlighting to the online versions of all posts:

This comes from a recognition that almost all browsers today can render the styles necessary for syntax highlighting.

Bringing styled code blocks to email posts

If possible, I’d like to render the code blocks in email posts as effectively as they’re rendered in the browser. Because Ghost is open source, it should be possible to customize emails enough to achieve this.

Like many open projects, Ghost has an API. Once a post is drafted, you can manipulate it programmatically through the API. Over the past week, I’ve been working on a script that does the following:

• Retrieve a draft post from my instance of Ghost;

• Convert every code block to an HTML block;

• Parse the contents of each code block, and apply inline styles using Pygments;

• Push a copy of the modified post back to my instance of Ghost.

It’s a bit more complicated than that, and if it ends up working I’ll write a post that covers the entire process in detail. But so far, it seems to be working. Here’s the same code block shown earlier, after being run through this extra processing:

This is a screenshot of a test email sent from Ghost. The email contains an actual code block, not an image of highlighted code. You can copy the code straight from the code block, and paste it into an editor. I believe screen readers can parse the code as well as they parse any code listing online.

Some newsletter writers are embedding images of code blocks in their emails to achieve this same effect. But that approach requires clicking a link to an external site if you want to work with the actual code. The image-based approach is also much less accessible, as screen readers can’t parse the actual code. You can’t enlarge the text or do any other manipulation that depends on access to raw text content.

Taking it one step further: Highlighting lines

One of the most important things to communicate in a technical post is how code changes as you progress from a simple example to a more complex, fully implemented program. On Substack, you have to rely on something like bold text for that.

What most writers want to do is highlight the lines that have changed in a listing. I’m trying to add one more feature to the Ghost processing script. It will examine the first line in each listing, looking for a directive such as hl-lines=[1, 3, 5,6,7]. If it finds a directive like this, it will insert additional styling to highlight these lines.

I almost have this working. One of the challenges is that code blocks look something like this when represented as strings:

from pathlib import Path\n\npath = Path("coffees.txt")

After being run through a highlighter like Pygments, they look more like this:

<span...>from</span> <span...>pathlib</span><span...>import</span>
 Path\n\npath <span...>=</span> Path(<span...>"coffees.txt"</span>

Basically, every element that needs to be styled is wrapped in <span></span> tags.

To highlight individual lines, you need to insert new tags at some of the line breaks. But syntax highlighters that don’t deal with lines as individual units don’t care where newline characters end up. After running text through a highlighter, you end up with line breaks in the middle of sections that have consistent styling:

<span...>import</span> Path\n\npath <span...>...

At one point I was trying to parse this kind of text. I needed to preserve the styling that Pygments had generated, and add my own style to highlight some lines. Here’s what I saw when I ran the parser:

That’s a lot of paths! And a lot of read_text() calls as well. :)

There are a number of things to notice here. First of all, as a programmer, I’ve always been amused to see output like this. I imagine many non-programmers think we just write some code and it either works or doesn’t work. I don’t think most people realize we see half-broken output like this on a regular basis.

This kind of output is also a little deflating, because as programmers I think we tend to be optimists. Whenever we run a program, we hope to see correct output. Seeing incorrect output is a bit of a letdown, because it means we’re not done yet.

But there are glimmers of hope in the output shown here. Some lines are highlighted, and the highlighted section contains some of the code I wanted to see highlighted. Also, there seems to be a pattern in the incorrect output. All the content we want to see is there, but some of it is repeated. There’s almost certainly a systematic mistake I’m making, that once corrected will generate the correct output.

The value of utility functions

I haven’t implemented a fix quite yet, but I have a good idea about what to try next. My problem arose in part because Beautiful Soup manages elements of a web page as a tree, made of nodes called tags. Some of the newline characters were embedded within a single tag, and I tried to split those tags up into several independent parts. I tried making copies of the tags and then adjusting the strings associated with each one. However, the new tags all pointed at the same string attribute that the original tag had. Changing the content of any one tag changed the content of the original tag, and all the new tags that had been created as well. That’s where the repetition in the rendered code block came from.

My post parser is a new project, and right now it’s a big mess of parts that mostly works. Rather than continuing to try to troubleshoot this issue within the context of the whole project, I’m going to pull the tag-splitting work out into a separate utility function. The function will take in a tag with embedded newlines, and return a sequence of tags with the newline characters isolated in individual tags. I’ll write a test for the new function, and then call the new function from the main project. This is the kind of work I was planning to do later, but doing it now will make my life a little easier today, and save future me some refactoring work.

If you can isolate problematic code, it’s almost always easier to troubleshoot, easier to use in a larger project, and easier to maintain as you run into more edge cases.

Conclusions

Most code blocks in emails look about the same as they did in the early 2000s. Hashnode is one of the first platforms I’ve seen that has rejected the idea that emails can’t contain nicely styled code blocks, and I appreciate their work in this area. Modern code blocks are more visually appealing, but more importantly they support clear communication about technical topics.1

I’m hoping to send next week’s newsletter out from Ghost, and it should have much better code blocks than what you’ve been seeing previously on Mostly Python. If you see better code blocks in the next few posts, you’ll have a good idea of what kind of work has gone into making them possible. :)

Note: When the first post goes out from Ghost, I’m still planning to send a final post from Substack shortly afterwards to let people know it’s been sent, and how to reach out if you can’t find it.

As much as I’ve enjoyed writing about Python and other technical topics, I’ve grown quite dissatisfied with Substack as a hosting platform. Starting next week, I’ll be using Ghost as a hosting service. The content of this newsletter won’t be changing, but the look and feel will be a little different.

In this post I’ll share some of the reasoning behind this change. I’ll also share some thoughts about how this relates to the choices programmers have to make about the platforms and services we rely on for many real-world projects.

What’s wrong with Substack?

I started using Substack because I was looking for a simple newsletter platform that would let me focus on writing. That’s what Substack was in its original form, but now they seem to be focusing on growth for growth’s sake. They also seem to be evolving into a social media platform that thrives on the kind of conflicts and divisiveness that has driven people away from other platforms.1

I just want to write a good technical newsletter, and Substack is no longer the most appropriate platform on which to do that.

Why Ghost?

If you haven’t heard of it, Ghost is a platform similar to what Substack was when it started. Just like with Substack, you can write a newsletter that becomes an online archive. You can write posts that go out over email, and others that are only available online. Ghost has been around for over ten years, and hasn’t tried to implement the kinds of social features that Substack has.

One of the strongest reasons to move to Ghost is that it’s built on top of open source software. The names are a bit confusing; there are several things people mean when they say “Ghost”:

• The open source project Ghost, which is a framework for “modern publishing”. It supports blogging, newsletters, and managing free and paid content. Because the core framework is open source, you can either self-host your publication or use a managed service.

• The managed hosting service Ghost Pro.

• The non-profit organization Ghost, which operates the Ghost Pro managed service. Revenue from operating the managed service supports ongoing development of the open source framework.

I’m going to use the managed version, because I want to focus on writing without having to do a bunch of sysadmin work. Because of its basis in open source, I feel much more confident that Ghost will be a long-term home for Mostly Python than I ever felt with Substack.

At this point it’s hard to choose a closed platform if an equivalent platform is available that’s built on open source. I’ve watched so many closed platforms and services take one or more of the following paths:

• Introduce ads in an effort to monetize user-generated content;

• Adopt dark patterns that pull users into higher usage levels than they originally intended;

• Add features that aren’t aligned with the original purpose of the platform or service, and slowly push people into using those features;

• Close off features and services that were central to the original platform’s appeal.

If Ghost Pro happens to fall into any of these patterns, there are two possibilities other than just moving to an entirely different platform. Since Ghost itself is open source, anyone can build a competing hosting service that leaves out the problematic features. Migrating between platforms that use the same underlying framework is much easier than migrating between different proprietary platforms. If a reasonable managed service doesn’t appear, I can always choose to self-host the service. I don’t particularly want to do that, but I may choose that option over migrating yet again Again, I don’t think I’ll need to move off of Ghost because it’s not susceptible to the same kinds of pressures that Substack is.2

The bigger picture

Most people reading this post probably aren’t planning to start a newsletter anytime soon. But many people are making decisions on a regular basis about which services to pay for, and which service providers to trust. When you’re working through these kinds of decisions, consider each of the following questions:

Can you export your critical data?

I knew Substack might not be a long term fit, but I was willing to give it a try because they allow you to export all your critical data. Writers “own” their content and subscriber list, and can export those at any time. Platforms that support exporting, and allow you to retain full ownership of your data, make it easy to implement effective backup routines and give you a way out of vendor lock-in.

Make sure you run an export early on, and look at exactly what’s in that export. While “you own your data and you can leave at any time” sounds nice, it’s probably a lot of work to process that data in a way that it can be used on another platform. Also, platforms don’t always keep their export services fully up to date. You might rely on a platform’s export function, and then when it comes time to use that data (for backup or migration purposes), find that some critical elements were never included in the export.

What’s the platform’s business model?

Every platform and service costs money to run. If the platform is charging enough for its core services, it’s more likely that they’ll be able to continue offering those services without trying to push unwanted “features” on the user base.3

How have they funded operations so far?

Understanding a platform’s funding history can give you a sense of how reliable their service is likely to be. For example, Substack has raised on the order of $100 million in venture capital. Now they need to generate enough revenue to pay that original investment back, while covering current operating expenses, and making a profit on top of that. If they can’t do that, they’ll need to seek even more funding, get acquired by a larger company, or fold. Sometimes a company will drift from its original mission in pursuit of a business model that’s profitable enough to pay off their earlier investment rounds.

The fact that a company has sought venture funding is not a bad thing in and of itself. It’s one way that promising companies can grow quickly, while they’re still establishing sustainable revenue streams. If funding levels align with a valid business model, the company may become sustainable. If a company has grown rapidly without a clearly sustainable business model however, that company’s platform can be much less reliable in the long term.

If you’ll only need a company’s services for a short time, none of this matters too much. But if you’re planning to rely on a service over a period of years, considering these kinds of details can help you make an informed decision about which platforms to trust.

What’s the public perception of the company?

In some areas, public perception doesn’t mean a whole lot. Nobody really likes Oracle, but that company is probably going to be around for a long time. People have serious frustrations with Adobe’s subscription plans, but their services are pretty reliable if you choose to use them.

A platform that requires a large user base, who could leave at any time, is a much different story. Platforms collapse and get acquired on a regular basis. Sometimes that plays out as a company closing suddenly, and sometimes the company does things over time that drive more and more users away. If the platform you’re choosing is susceptible to these dynamics, look at some of the public statements its leaders are making. Try to get a sense of whether they might be creating problems for themselves in the near to mid term future, or if they inspire confidence in their leadership and decision-making abilities.

Every time Substack has experienced conflict around their policies, they’ve doubled down on the policies that led to those conflicts in the first place. I’ve spent a good part of my life dealing with conflict, and I’m happy to address it when necessary. But when a company or platform fosters conflict because it increases engagement, I want no part of that.

Conclusions

I’ve really enjoyed writing Mostly Python over the past year, and look forward to many years of continuing to do so. Next Thursday’s post will have the same kind of content you’re used to seeing here; it will just look a little different.

I’ll send one last email out from Substack after publishing from Ghost for the first time. That email will include a way to reach out if something goes wrong in the transition. If you’re a paid subscriber, Substack and Ghost both use Stripe for billing, so your subscription should carry over smoothly.

Thank you for joining me here on Substack, and I hope you’ll stay with me through this transition as well.

I learned about testing fairly late in my career as a programmer. It wasn’t introduced in any of the classes I took when I was younger, and the intro and intermediate books I read over the years didn’t mention it in any prominent way. As soon as I learned the basics of testing, I wished I had learned about it earlier.

Testing is often thought of as something separate from development, but it doesn’t have to be. Testing a project effectively is an interesting challenge, and often pushes you to understand your project in new and meaningful ways.

I’ve had the good fortune to get to maintain a comprehensive introductory Python book for almost a decade now. That’s a time period where building out a thorough test suite is worth the effort. The challenges that came up in this work highlight some aspects of testing that don’t necessarily come up when developing a test suite for a more traditional software project. In this final post I’ll highlight some of these takeaways.

General takeaways

There are a number of big-picture takeaways from this work:

Think about your testing constraints

When you’re in charge of writing a test suite, you end up looking at your project in a different way. Instead of thinking about the people who use your project, you have to write a program that uses your project. If your project wasn’t designed to be used in a fully automated way, you’ll have to find a way to exercise the code from within the test suite. Sometimes you can change the actual project to make it easier to test, but sometimes there are very good reasons to keep the project as is, and come up with a test suite that handles the current structure of the project.

Code for a book is a perfect example of this kind of situation. When someone writes code for a book, they tend to prioritize simple examples that highlight the topic that’s being taught. Often times this means leaving out some elements that we’d include in a real-world project, which would distract people from learning about a specific topic.

In a more traditional project, you might have to test code that’s already being used in critical ways, or by a significant user base. It might be necessary to test the code as is, before making any changes just for the purposes of testing. Learning to recognize the constraints you’re working with is an important skill in testing.

Think about possibilities

All that said, you can be quite creative in developing your test suite. As an author, I can’t change any of the code I’m testing. But I can copy that code to a new repository or a temp directory, and do anything I want with it from that point forward.

In many testing tutorials and references, we work with code that generates text output. It’s often somewhat straightforward to verify whether text output is correct. However, most real-world output is more varied than that. If you’re working with text output, maybe there’s some elements of the text that’s always changing, such as timestamps. Maybe you’re working with image output, or HTML, or even sound files. It’s not always obvious how to verify that non-text output is “correct”.

I think a lot of testing resources talk about assertions where all output is either correct or incorrect. But I’ve run into many situations, especially when testing cross-platform code, where this is the actual question that needs to be asked:

Is this output close enough to what it’s supposed to be?

It’s not particularly difficult to write assertions that check whether output is “good enough”, but I don’t see it covered all that often. For example when testing output in the form of a plot image, it was satisfying to come up with an approach that looks at the average color values of the entire plot, rather than trying to make assertions about the actual data represented in the plot. I have no idea if this is the best approach to testing graphical output that can vary over time, but it’s certainly good enough for my purposes at this point.

Think about dependencies

The standard view of testing is that we should test against all versions of the language that the project currently supports, and multiple versions of its main dependencies. But this dependency matrix can grow quite large. Although there are tools that help you manage testing against that entire matrix, you don’t always need to test against the entire matrix.

In the context of testing code from a book, I’m primarily interested in testing against release candidates of new versions of Python, and major libraries such as Pygame, Matplotlib, Plotly, and Django. I don’t need to test against a full matrix of different versions of these libraries; instead I need to be able to name the specific version that I want to test against during the current test run.

I’m working on a deployment-focused project that probably doesn’t need to maintain backwards compatibility as carefully as many other projects. This project should only be run once against any given target project. It shouldn’t be an issue for someone to just install the latest version and then carry out their deployment. The test suite for that project is being developed with a focus on testing the current version of the library, and probably won’t do extensive testing against previously-released versions.

Think about test artifacts

This point deserves a post, or even a series, of its own. A test suite runs your project in various ways. As such, it can create any artifacts needed for automated testing, and any artifacts that you might want to open and interact with manually.

For example, it might be possible to write a plot image to memory instead of to a file, and avoid the overhead of reading and writing files during a test suite. But I sometimes want to go to the temp directory used by the test suite, and open up plot images to look at them. Even if tests pass, I might want to see what the plot looks like under current testing conditions. If a test fails I might want to look at the image that was generated during the test run, instead of having to run the project manually as part of my debugging work. I’ll say this over and over again, a good test suite can act as a development tool, not just a verification tool.

Think about goals

For people who haven’t done a lot of testing work, the perception can be that testing is entirely about the binary question “Does this code work correctly or not?” But a well-developed test suite should answer a specific set of questions that you have, which meet your goals in developing and maintaining the project:

What parts of this project work as expected?

What parts are starting to work differently than expected?

If part of this project is behaving differently than it used to, what is that difference?

For example if a page in a web project fails to load, is the error on that particular page, or was there an error in a different part of the project that indirectly affects other pages? When this kind of bug occurs, will your test suite lead you efficiently to the root cause, or will it focus on the symptom of the underlying problem? I haven’t always known that a test would help me in this way, but I’ve certainly seen some testing approaches generate more helpful information than others.

There’s a tendency to think that tests should run as fast as possible. But we’re actually after a balance between speed, test coverage, and information generated. If you focus exclusively on speed, you might miss out on generating some information and artifacts that can be really helpful in debugging and development work. If you generate too much information, you might make it difficult to use the output of your test suite.

Make sure your tests can fail

Sometimes we focus so much on the desired outcome of tests passing, that we forget to make sure a test can fail. If you haven’t seen a failure in a while, consider introducing an intentional bug to see if the expected failures appear.

pytest-specific takeaways

pytest is a fantastic library. It’s one of those amazing libraries that’s simple enough in its most basic usage that people entirely new to testing should start using it. At the same time, it’s flexible and powerful enough that it’s perfectly appropriate to use in large, fully-deployed commercial projects.

You can get started with pytest from a simple tutorial. But there are lots of specific things to learn about it that will help you write exactly the kind of test suite that suits your needs. Also, there are a number of ways to run your tests, so you can use them in exactly the way you need at any given time.

Use parametrization

At first glance, parametrization seems like a way to write less code. But that’s just one benefit. The real power of parametrization comes from developing a highly consistent way to run a series of similar tests. In the context of this test suite, two test functions resulted in 60 tests being run.

Once you’re familiar with the concept and syntax of parametrization, it lets you generate a large number of tests by writing a small number of test functions. While appearing more complex to people unfamiliar with parametrization, it’s actually a much more maintainable way to develop a growing test suite.

Use fixtures

Fixtures are used to carry out the setup work required for test functions. Fixtures can do something, such as building a virtual environment for a set of tests. They can also return resources needed for tests. By defining a fixture’s scope, it can be run once for the entire test session. Or, it can be run for specific modules, classes, or functions.

As with many features of pytest, you can start out by writing a simple fixture that carries out a small task or creates a small resource for several test functions. As your experience and understanding grows, you can use fixtures in increasingly complex and powerful ways.

Use parallel test execution

By installing the pytest-xdist package, you can run many kinds of tests in parallel. If your tests are starting to take a noticeable amount of time to run, see if a simple call to pytest -n auto speeds up your test run.

Parallel tests don’t always work well if your tests access external resources, or have complex setup steps. But you can install pytest-xdist and run a subset of your tests in parallel, and run tests that don’t work well in parallel separately.

Use custom CLI args to enhance your test suite

In this series I used a set of custom CLI args to specify exactly which version of a library to use during the test run. This is particularly useful when testing against release candidates of third-party libraries.

You can use CLI args in all kinds of ways. For example, imagine your test suite generates a large number of image files and then compares those against a set of reference images. An update to a library you use changes the output images slightly, but in an acceptable way. You could go into your test’s temp directory, check the images, and then manually copy them over to your reference_files/ directory. But if you do this on a regular basis, you could write a CLI arg that lets you run a command like this:

$ pytest tests/test_image_output.py —-update-reference-files

You could then write code that copies each newly-generated image file to reference_files/ when this flag is included.

Learn to use pytest’s built-in CLI args

pytest has a significant number of CLI args that you can use to run exactly the tests you want, generating exactly the kind of output you want on any given run. For example, here are a few of my most-used arguments:

• The -k argument lets you specify tests matching a specific pattern. This can include test function names or parts of names, and parameters as well.

• The -x argument stops at the first test failure.

• The —-lf argument re-runs the last test that failed.

• The -s flag tells pytest to show the output that was captured during test execution.

• I recently learned about the —-setup-plan argument, which shows exactly what pytest will run, in what order. This doesn’t just show what order the tests will be run in. It also shows when each fixture will be called, and how often they’ll be called. You can see how it runs fixtures before test functions, and how it uses parametrization to repeat tests with different parameters. It’s an amazingly helpful tool for understanding more complex aspects of pytest.

Understand the role of multiple conftest.py files

You can write a conftest.py file in any directory within your test suite, including your project’s root directory. This lets you structure your test suite efficiently. For example if a fixture is used by multiple modules, you can write that fixture once in a conftest.py file in the directory containing those modules. You don’t need a copy of the fixture in every test module.

Some people don’t like seeing multiple conftest.py files, because it can be less clear about where fixtures used in a test function are coming from. A good principle to minimize confusion is to place fixtures as close as possible to the tests that use them. For example if you have tests in a subdirectory, put the fixtures that are only used by those tests in that directory’s conftest.py. Don’t dump all of your fixtures into your root directory’s conftest.py file.

Minor takeaways

If you’ve made it this far, you must be interested in testing! Here’s a few more smaller things that are worth mentioning:

Consider making assertions about your tests

At the heart of every test is an assertion about your project’s behavior or output. But if there’s any complexity to your test suite’s setup, consider writing assertions about that setup work itself. This is especially important to consider if there’s a chance the setup work might not be completed correctly, in a way that would still allow the tests to pass. Or, if a step in the setup work fails you might want to just exit the test suite and deal with that setup failure.

Consider modifying a copy of your project’s code

Some projects are more “testable” than others. If your code seems untestable as currently written, you should probably restructure your code so it’s easier to test the overall project and its component parts. But if you can’t modify your code for some reason, consider making a copy of all or part of the project, modifying the copy, and then making assertions against the modified code. For example you might modify your code to write output to a file instead of displaying it on the screen. Then you can make assertions about the contents of that file.

Consider testing a subset of your project’s codebase

Chasing “100% test coverage” can be an exercise in over-optimization. There may well be other things that are more worth pursuing than complete test coverage of your codebase. Identify the most critical aspects of your project, and make sure to test those behaviors and sections. Also, if some sections of code are repetitive and a successful test implies correct behavior for the repeated parts, ask yourself if it’s reasonable to only test one or several of the repeated behaviors.

Don’t refactor just to refactor

Most of us are taught to refactor code that appears multiple times in a project. In a test suite, that’s much less important. Repetition is okay in testing, because it’s often helpful to have enough context in a test function to assess a test failure without having to trace through a complex hierarchy of code. Refactor your tests as needed to understand and maintain the evolving test suite, but don’t refactor just because you see repetition in your tests.

Use a seed to test random code

If your project uses randomness at all, consider using a random seed to get repeatable pseudorandom output.

Be wary of metadata, and other minor file differences

If you’re comparing newly-generated files against a set of reference files, be wary of metadata. In several projects now, I’ve spent a lot of time trying to figure out why two seemingly identical files fail a comparison test. Sometimes it’s a piece of metadata generated by a library used in generating the output file. Sometimes it’s a minor difference in how output is written on different operating systems. If files appear identical but fail a comparison test, figure out how to view the metadata on your system and take a look at it for your test file and your reference file.

Write your tests on one OS, and then generalize to others early

If your test suite will need to pass on multiple operating systems, write a small number of tests that work on your main OS. Before expanding your suite too significantly, run those tests on the other OSes you need to support, and figure out how to make them pass there as well. You can save yourself some refactoring work and some complexity by discovering cross-platform issues early on. You might learn more about how your project and its dependencies work on different OSes as well.

Conclusions

There are a lot of tutorials out there about testing example projects. But testing a real-world project often brings up challenges that don’t come up in clean sample projects. Most people reading this series probably aren't writing technical books, but the issues that arose in developing this suite are comparable to the issues that people face when developing a test suite for a real-world project.

I love testing, because it helps me sleep better at night. If I’ve tested my code against a release candidate of a library, I don’t worry about a slew of bug reports when that new version comes out. If I have to put a project aside for a while, it’s easier to pick up work on that project months later, or even years later, if it has at least a basic test suite. When resuming work on the project, I can run the tests and see if anything has broken for any reason in that project’s down time.

I also love hearing people’s testing stories, and hearing the questions people have about testing. If you have stories about test suites that saved you at some point, or tests that were frustrating in some way, please share it. If you have questions about how to approach some aspect of testing, I’d love to hear that as well. If your question is better asked privately than in comments, feel free to reply to this email. I can’t promise to have an answer, but I’ll be happy to share any thoughts I have that might help.

The final project in Python Crash Course is a simple, but nontrivial Django project. It’s called Learning Log, and it lets users create their own online learning journals.

When testing a typical Django project, the focus is usually on testing the internal code in the project. I’m not really interested in that kind of testing here. Instead, I want to know that this project works on new versions of Django, which come out every 8(!) months. That’s a quick lifecycle, but it’s manageable because Django takes stability quite seriously.

In this post we’ll develop a single test function for the Learning Log project. It will create a copy of the project, build a separate virtual environment just for testing this project, and test whether the overall project functions as it’s supposed to.

Setting up the test environment

A Django project really needs its own virtual environment. I want this test to reflect how readers run the Learning Log project, so the test is going to make a fresh environment just for this project on every run. It’s then going to take the same actions a reader would when interacting with the project.

Copy project files to temp directory

Here’s the first part of test_django_project.py:

Read more

In the previous post we tested the output of programs that use Matplotlib to generate plots. Most of that work focused on testing output images, but we also dealt with randomness and metadata.

In this post we’ll test programs that use Plotly to generate visualizations. We’ll use some of what we learned in the Matplotlib tests, but we’ll also run into some new challenges in this test suite. For example, what do you do when two 3.4MB HTML files differ by only 100 characters?

Rolling dice

The first three programs we need to test simulate rolling a variety of dice. Here’s the structure of the first program:

# Create a D6.
die = Die()

# Make some rolls, and store results in a list.
results = []
for roll_num in range(1000):
    ...

# Analyze the results.
frequencies = []
...

# Visualize the results.
title = "Results of Rolling One D6 1,000 Times"
labels = {'x': 'Result', 'y': 'Frequency of Result'}
fig = px.bar(x=poss_results, y=frequencies, title=title, labels=labels)
fig.show()

This program creates a model of a six-sided die, makes 1,000 rolls, and generates a histogram of the results:

The fig.show() method writes an HTML file to a temporary location, and opens that file in a browser. In the test function, we’ll replace fig.show() with a call to fig.write_html(), and then work with that HTML file directly.

Testing die programs

In a new test module, test_plotly_programs.py, we’ll first copy the files we need to a temp directory:

Read more

In this post we’ll see how to deploy a simple project like this to Streamlit’s Community Cloud hosting service. It’s a fairly straightforward process, but there are a couple privacy issues you might want to be aware of.

What is Community Cloud?

Streamlit refers to projects as apps, because once you have a working project it functions like an app. I’ve been talking about dashboards, but Streamlit is flexible enough that what you create doesn’t have to look like a dashboard.

When you’ve built a functioning app with Streamlit, you’ll almost certainly want to deploy it somewhere so others can use it as well. Because Streamlit is an open source project that can be installed with pip, it can be deployed in the same way you deploy any Python code: on a physical server, on a VPS such as Digital Ocean or Linode, or on a managed Platform like Heroku.

If your app is hosted in a public repository on GitHub, you can deploy it for free to Streamlit’s Community Cloud service. There are a variety of reasons companies offer free hosting for public projects. For one thing, having a collection of open projects helps people learn how to use the underlying framework. The code for every project hosted on Community Cloud is one click away; if you see an app or feature you like, you can see exactly how its developers implemented the project using Streamlit.

You can make an account on Community Cloud using Google, GitHub, or through email-based authentication. If you don’t already have an account, I encourage you to read through the rest of this post before making one. Streamlit asks for more permissions than you probably want to grant, and there’s a way to avoid some of the permissions that isn’t very obvious.

Connecting a GitHub repository

I pushed the code for the Rolling Dice project to a GitHub repository, so it’s ready to deploy to Community Cloud. Make sure your project has a requirements file, so Streamlit will know what packages it needs for deployment. To deploy a public project, go to https://share.streamlit.io.

I already have a Community Cloud account, but I disconnected my GitHub account in order to walk through the process again. Without a connected GitHub account, this is what the share.streamlit.io dashboard looks like:

Notice the caution symbol next to Settings. Streamlit really seems to want access to your GitHub repositories, and if you limit the access in any way you’ll see that symbol.

If you click New app and you haven’t already connected your GitHub account, you’ll see a button labeled Connect to GitHub:

There’s no way to avoid this connection, so click that button.

You’ll arrive at a GitHub page where you can authorize Streamlit to access your public repositories:

I don’t particularly like the permissions that Streamlit asks for. They want full admin and read/write access to all your public repositories. I think this is because they offer an in-browser editor where you can work on your Streamlit projects. But they don’t give the option to opt in to that specific service; if you want to deploy a public project to Community Cloud, you have to grant all these permissions.

Public repositories are part of the open source supply chain, and asking for full access to all of a user’s repositories seems a little inappropriate in 2024. Streamlit says they don’t want to have to ask for permission more than once, but most developers I know are perfectly fine granting permissions on a per-repo, as-needed basis.

I believe you can grant these permissions, deploy your project, and then revoke permissions. You’ll lose access to some services through Streamlit until you grant permissions again, but your projects appear to continue running.

If you see a request for access to your private repositories as well, you can bypass that request by going directly to the URL https://share.streamlit.io/deploy. I believe with a new account Streamlit tries to get access to private repositories. I bypassed that once, and it doesn’t seem to prompt for private access on subsequent deployments.

Deploying your project

After dealing with permissions, you’ll arrive at a page where you can tell Streamlit which repository you want to deploy from:

You can fill this page out with your repository’s URL, the name of the branch you want to deploy from, and the name of the .py file that you’ve been calling with streamlit run. Streamlit will generate a unique URL for your project, and you’re free to enter a simpler name that hasn’t already been taken. For this project, I chose sl-roller.streamlit.app.

When you click Deploy, you’ll see a message that “Your app is in the oven.” In a moment, you should see a deployed version of your app that behaves just like the one you’ve been running locally. Here’s what the Rolling Dice demo looks like immediately after deployment:

When you’re logged in to Community Cloud, you’ll see a sidebar that lets you manage your app. You can view the log, see how many people have visited your app, reboot it as needed, and more.

A little more about private repos

Streamlit really seems to want access to our private GitHub repositories. I bypassed the popup that demands access to private repositories as described above, but I still have a caution symbol next to my Settings link. Here’s what I see if I click on Settings:

Streamlit’s justification for this warning is that they allow you to deploy one app from a private repository. But if you click Allow access, this is what you see:

For the possibility of deploying from one private repository, Streamlit wants read and write access to all of your private repositories. In an issue on the streamlit repository, staff have suggested that people can make a new GitHub account if they dislike Streamlit’s approach to handling permissions. This is not a very satisfying response, as it shifts all the responsibility for appropriate access onto developers.

I’m only using Community Cloud because it’s possible to bypass the demand for private repository access. If you decide to use it as well, I encourage you to keep that caution symbol next to your Settings link as well.

Pushing changes

When you connect a GitHub repository to a Community Cloud deployment, Streamlit sets a web hook on the repository. Any time you push changes to the branch that the deployment is pulling from, Streamlit gets a notification and automatically updates the deployed project.

This means when you push updates to your repository’s main branch, or merge changes into the main branch, your deployment will pick up those same changes.1

Conclusions

Streamlit is fairly easy to get started with, and if you’re already familiar with Git and GitHub it’s quite easy to deploy your public projects as well.

Be aware that projects hosted on Community Cloud have about 1GB of resources available, and they rate-limit requests to prevent abuse. I shared a project that got several requests a minute for a while, and the app handled it perfectly fine. I shared that same project to a larger audience, and everyone started getting 429 errors. If you have a project that’s going to see many concurrent users, Community Cloud is probably not an appropriate deployment solution.

I’ll follow this up with a couple more posts about using Streamlit over the next month or two. I’ll share a real-world dashboard I developed, and discuss a variety of issues involved with making performant deployments of your Streamlit apps. If you have questions or observations from your own experiences, please share them!

Resources

You can find the code files from this post in the roller_demo GitHub repository.

In the last post we tested a game built with Pygame, and added a CLI argument to specify any version of Pygame we want. In this post we’ll test the output of data visualization programs that use Matplotlib. In the end, we’ll be able to run tests against any version of Matplotlib as well.

Testing data visualization programs is interesting, because there’s a variety of output. There’s some terminal output that’s fairly straightforward to make assertions about, but the graphical output is much more interesting to test. Let’s dig in.

Testing one Matplotlib program

Here’s the first program we want to test, called mpl_squares.py:

import matplotlib.pyplot as plt

input_values = [1, 2, 3, 4, 5]
squares = [1, 4, 9, 16, 25]

plt.style.use('seaborn-v0_8')
fig, ax = plt.subplots()
ax.plot(input_values, squares, linewidth=3)

# Set chart title and label axes.
...

plt.show()

This code defines a small dataset, and then generates a line graph:

The call to plt.show() causes Matplotlib’s plot viewer to appear. This is great for introducing people to plotting libraries, but it’s problematic in an automated test environment. We don’t really want to call plt.show() in a test run.

Rewriting the file before testing it

We won’t test this file directly because we don’t want to call plt.show(), and we can’t change the files in the book’s repository. However, we can copy them to a different directory outside the repository and then do whatever we want with them. So let’s modify the program to write the plot as an image file. We can then make assertions about the freshly-generated image file.

We’ll start by making a new test file, called test_matplotlib_programs.py. Here’s the first part of the test for mpl_squares.py:

Read more

In the last post we focused on testing the basic programs in the first half of Python Crash Course. So far the test suite covers 60 programs, which only print output to the terminal. The tests run on macOS and Windows, and should run on most Linux systems as well. This first phase of work brought up some interesting testing concepts such as parametrization and fixtures, but the code that was being tested was pretty simple.

In this post we’ll make sure the test suite can be run with multiple versions of Python. After that, we’ll test the first project in the book, Alien Invasion. This is a 2d game project, and it’s probably not obvious at all how to test it. We’ll discuss those challenges, make a plan to address them, and write a test that gives us confidence that the game project works as it’s supposed to.

Running tests on different versions of Python

Note: I drafted this post last fall, before Python 3.12 was released. I’m leaving the version references as they were before 3.12 came out, because it shows how the test suite can be used during the pre-release period for new versions of Python.

Most of my Python work at the moment uses Python 3.11.5, the most recent point release of the latest version of Python. The test suite runs in a virtual environment that was built with this version.

However, Python 3.12 is coming out soon, and the third release candidate (3.12.0rc3) was released about a week ago. I’d like to test the book’s code against this version, and each release candidate that comes out until 3.12.0 is officially released. This helps find any bugs in the pre-release versions of Python, and lets me address any incompatibilities in the existing codebase with newer versions of Python.

I use pyenv to manage Python versions on my system. 3.12.0rc3 isn’t available quite yet through the Homebrew version of pyenv, so I’m going to use 3.12.0rc2 for this round of testing.

To make testing with different versions easier, I’m going to build a new virtual environment called .venv_3120rc2, and run the tests with that environment active. Here’s the terminal commands I’m using to switch between different Python versions, and verify that the commands are having the expected effect:

(.venv)$ python --version
Python 3.11.5
(.venv)$ deactivate
$ pyenv local 3.12.0rc2
$ python --version
Python 3.12.0rc2

Now we can make a new virtual environment, and run the tests using that version of Python:

$ python -m venv .venv_3120rc2                             
$ source .venv_3120rc2/bin/activate
(.venv_3120rc2)$ pip install pytest
(.venv_3120rc2)$ pytest -q
... [100%]
60 passed in 0.94s

This shows that all the basic programs in the book should continue to run without issue under Python 3.12.

Note that this did not involve any changes to the test suite itself. The test suite uses the Python interpreter from the active virtual environment, so all we did was set up a second virtual environment. We can keep using the 3.12.0rc2 environment, or we can switch back to the 3.11.5 environment whenever we want.

Making it clear which version was used for testing

Dealing with multiple Python versions is not always straightforward. When you have multiple versions installed, it’s easy to end up accidentally using a different version than what you intended. This is especially true if everything works, and there’s no version-specific output.

Let’s add a short block that shows exactly which version was used to run the tests. We’ll do this in tests/conftest.py:

...
@pytest.fixture(scope="session")
def python_cmd():
    """Return the path to the venv Python interpreter."""
    return utils.get_python_cmd()

def pytest_sessionfinish(session, exitstatus):
    """Custom cleanup work."""

    # Show which version of Python was used for tests.
    python_cmd = utils.get_python_cmd()
    cmd = f"{python_cmd} --version"
    output = utils.run_command(cmd)

    print(f"***** Tests were run with: {output}")

If you include a function called pytest_sessionfinish() in conftest.py, that function will be called automatically when the test session is over. We’d like to use the python_cmd fixture in this function, but fixtures are only available during a test session. By the time pytest_sessionfinish() is called, the test session is over. So, we need to pull the code for getting the value of python_cmd out into a utility function. This shortens the original python_cmd() fixture function, which effectively becomes a wrapper for the new utility function.

In pytest_sessionfinish(), we call python —-version, where python is actually the full path to the interpeter that’s being used throughout the test session. We then print a line showing what the output of this call is. Normally you need to include the -s flag when calling pytest to see the output of print() calls. That won’t be necessary here, because the output of any print() calls in pytest_sessionfinish() are included in the overall test output by default.

When we run tests using .venv, we should see output reflecting Python 3.11. When we run tests using .venv_3120rc2, we should see output referencing Python 3.12:

$ source .venv/bin/activate             
(.venv)$ pytest -q
... [100%]
***** Tests were run with: Python 3.11.5
60 passed in 0.90s
(.venv)$ deactivate 
$ source .venv_3120rc2/bin/activate
(.venv_3120rc2)$ pytest -q
... [100%]
***** Tests were run with: Python 3.12.0rc2
60 passed in 0.96s

The output here shows one test run using Python 3.11.5, and a second run using Python 3.12.0rc2.

This actually caught an inconsistency for me while doing this writeup. I sometimes install pytest to my global Python environment, so I can run quick tests anywhere. But if you then install pytest to a virtual environment, it can be unclear at times which instance of pytest is being used. I ran pytest in an active 3.12.0rc2 virtual environment, but the output indicated Python 3.11 was being used. Even with the virtual environment active, pytest was still pointing to my system-level instance of pytest, because it was the first one listed in my path variable. I ended up uninstalling that system-level pytest, to avoid any future inconsistencies like this.

Running pytest without the -q flag usually shows which interpreter was used to start the test suite, but I always want to know that python_cmd is picking up the version I think it is.

Why not use tox?

The tox library automates testing across different versions of Python, and different versions of third-party libraries as well. You might be wondering why I don’t use tox to manage versions instead of making individual virtual environments, which is similar to what tox does for you behind the scenes.

I haven’t used tox before, but I’m looking forward to using it when I have a need for it. I’m not using it for this test suite because I usually want to run tests against a specific version of Python, or a specific version of a library. I rarely want to run a full matrix of all versions of Python with all versions of each library. For my testing purposes, most of the time I want to pick a specific version of Python and maybe a specific version of a library. This is another way that testing for a book differs from testing traditional software projects.

Testing Alien Invasion

Testing video games is notoriously difficult. Most discussions of testing software focus on exercising code which has identifiable inputs we can test, and outputs we can make assertions against. The output of video games depends on a variety of inputs that often can’t be replicated fully. That’s why game companies pay people to play through their games in specific ways, and report on their experiences. It’s not nearly as fun as it sounds to young players.

Fortunately my testing needs are simple: I want to make sure the game runs on recent versions of Python and Pygame. The original game in the first edition of the book was function-based, and it would have been rather hard to write an automated test for the game. I restructured the game for the second edition of the book so that the entire game is represented by a class. This made the code a little more complicated at the start of the project, but resulted in a much cleaner overall structure for the finished game. It also made the game much more testable—you can import the main game file, create an instance of the game, and script the same kinds of actions an actual player might take.

A few years ago I wrote up a guide for writing a script that would play the game automatically. It’s kind of fun to write a program that plays a game for you. Alien Invasion is a classic space shooter, so you can implement algorithms like “move the ship back and forth, firing continuously until the screen is cleared”. You can also write algorithms that target specific aliens, or algorithms that combine multiple strategies and include random fluctuations in behavior.

This background work made it fairly straightforward to figure out how to test Alien Invasion. The test function will call a script that automates the game play. The automation script will implement a simple strategy that clears one screen of aliens, and then exits. We’ll then make a few assertions about certain conditions that should be true at the end of Level 1. If those simple assertions pass, I’ll be confident that the game project works on the versions of Python and Pygame that were used for testing. I have no need to test more complex aspects of the game itself.

Automating gameplay

Here are the most important parts of ai_tester.py, the script that automates game play for testing purposes:1

class AITester:

    def __init__(self, ai_game):
        """Automatic player for Alien Invasion."""
        # Make a reference to the game object.
        self.ai_game = ai_game

    def run_game(self):
        """Replaces the original run_game(),
        so we can control the game.
        """
        # Start out in an active state.
        self.ai_game.game_active = True

        # Speed up the game for testing.
        self.ai_game.settings.speedup_scale = 50
        self.ai_game.settings.increase_speed()

        # Start the main loop for the game.
        while True:
            # Still call ai_game._check_events(),
            # so we can use keyboard to quit.
            self.ai_game._check_events()

            # Sweep the ship right and left continuously.
            ship = self.ai_game.ship
            screen_rect = self.ai_game.screen.get_rect()

            if not ship.moving_right and not ship.moving_left:
                # Ship hasn't started moving yet; move to the right.
                ship.moving_right = True
                ...

            # Fire as often as possible.
            self.ai_game._fire_bullet()

            self.ai_game._update_screen()
            self.ai_game.clock.tick(60)

            if self.ai_game.stats.level > 1:
                break

Notice that AITester does not inherit from AlienInvasion. Instead, it takes a reference to the game object and then defines its own run_game() method. That method takes the place of the original run_game() method, so we can control all actions in the game.

In this automated player, we start the game in an active state instead of waiting for the player to press a start button. We speed the game up by a factor of 50 for testing. If you speed it up too much, the aliens come down too fast to run through any actual game interactions.

We then start the ship moving and sweep it left and right continuously. As long as the game is active, we keep firing bullets at the aliens. We break out of the game’s main loop as soon as we’ve finished the first level.

Testing the game

Now we need to write a test function that calls AITester. We’ll do that in a new test file, called test_alien_invasion.py:

from pathlib import Path
...

def test_ai_game():
    """Test basic functionality of the game."""

    # Add source path to sys.path, so we can import AlienInvasion.
    ai_path = Path(__file__).parents[1] / "chapter_14" / "scoring"
    sys.path.insert(0, str(ai_path))
    from alien_invasion import AlienInvasion
    from ai_tester import AITester
    
    # Create a game instance, and an AITester instance, and run game.
    os.chdir(ai_path)
    ai_game = AlienInvasion()
    ai_tester = AITester(ai_game)
    ai_tester.run_game()

    # Make assertions to ensure first level was played through.
    assert ai_game.stats.score == 3375
    assert ai_game.stats.level == 2

This is a fairly short test function; most of the work for testing the game is already in the AITester class. Here we modify sys.path to include the directory where alien_invasion.py is located. There are many versions of the game project in the repository, to help readers who get off track in the game’s development. These are versions of the project as it stands at the end of each major section in the book. We only need to test the final version of the game; if that works, then all the other versions should work as well.2

The main game file imports a number of other modules, which all depend on relative imports. So, we use os.chdir() to switch to the directory where alien_invasion.py is located. Since the game project doesn’t modify any files, we can safely run it from that directory.

Once in the ai_path directory, we can make an instance of AlienInvasion, and then make an instance of AITester. We call run_game(), and it automatically plays the game through one level.

When the first level is complete, we make two simple assertions: that the score is correct for having destroyed one screen’s worth of aliens, and that we’re at level 2. There are many more assertions we could consider, but that’s not really the point. The main point of this test is to show that the current code works for running the game, on the current version of Python and Pygame.

This test passes, and you can watch the AI player as it plays through the first level:

Interestingly, I slowed the game down in order to make a screenshot, and noticed the simple game play algorithm doesn’t actually clear the screen if it’s not sped up.

At full speed, the test finishes in about 6 seconds:

(.venv)$ pytest tests/test_alien_invasion.py -q
. [100%]***** Tests were run with: Python 3.11.5
1 passed in 6.39s

Six seconds is slow for a single test in most projects, but this is quite reasonable for what it does. It’s reassuring to see the game running for a few seconds, and I’m never going to use this in a CI environment. It’s also easy to exclude this test if I’m focusing on other tests.

Testing different Pygame versions

One of the main goals of this test suite is to be able to test different versions of a specific library. To support this, I’m going to add a CLI argument to the test suite, so I can run the following command:

$ pytest tests/test_alien_invasion.py --pygame-version x.y.z

Here’s the new function in conftest.py that adds this CLI argument:

def pytest_addoption(parser):
    parser.addoption(
        "--pygame-version", action="store",
        default=None,
        help="Pygame version to test"
    )

The pytest_addoption() function adds new CLI options to pytest calls. The call to parser.addoption() defines —-pygame-version as an individual argument. Any value passed with this argument will be stored, and accessible throughout the test suite.

We’ll use this argument in a fixture function in test_alien_invasion.py:

@pytest.fixture(scope="module", autouse=True)
def check_pygame_version(request, python_cmd):
    """Check if the correct version of Pygame is installed."""
    pygame_version = request.config.getoption("--pygame-version")

    if pygame_version:
        print(f"\n*** Installing pygame {pygame_version}\n")

        cmd = f"{python_cmd} -m pip install pygame=={pygame_version}"
        output = utils.run_command(cmd)

The code to check the version of Pygame needed for testing is placed in a fixture, for a couple reasons. First, this is setup work; it’s not really what we’re testing. That kind of work tends to belong in fixtures as opposed to test functions. But also, we’re going to do this same kind of thing when testing the other projects in the book. We’ll be using code similar to this later, and it will be easier to reuse this code if it’s implemented as a fixture from the start.

This fixture doesn’t need to return anything. When a fixture doesn’t have a return value, you can specify autouse=True, and the fixture will be called automatically for all tests in its scope. This fixture only applies to Pygame, so I’ve placed it in the test_alien_invasion.py file and given it the module scope.

You can check a custom pytest CLI argument by calling request.config.getoption(). If —-pygame-version was specified, we print a message that we’re installing a specific version, and then install that version to the active virtual environment.

Note that there’s no change needed to the actual test function, test_ai_game(). With an autouse fixture, the setup work is done automatically by the fixture.

We need to pass the -s flag in order to see the output of print() calls in fixtures and test functions. We can now test any version of Pygame we want. I’m usually interested in testing development releases, so I’ll specify a recent development release:

(.venv)$ pytest tests/test_alien_invasion.py --pygame-version 2.5.0.dev2 -qs
*** Installing pygame 2.5.0.dev2
pygame 2.5.0.dev2 (SDL 2.26.4, Python 3.11.5)
Hello from the pygame community...
.
***** Tests were run with: Python 3.11.5
1 passed in 19.24s

As the test suite runs, we can see that the requested version of Pygame is being installed. That information is verified in the output that Pygame generates when it’s loaded. We’ve modified the virtual environment in a way that will persist after the test runs, but we’ll address that in a moment. Note that this session takes longer than usual, because we’re installing a package and using it for the first time during the test run.

The check_library_version() utility function

We’re going to use this same approach for the data visualization and web app projects as well, so let’s move the code for checking library versions to a utility function. When doing small refactoring work like this, I like to write the function call I want to make, and then write the function itself.

Here’s the new version of the fixture in test_alien_invasion.py:

@pytest.fixture(scope="module", autouse=True)
def check_pygame_version(request, python_cmd):
    """Check if the correct version of Pygame is installed."""
    utils.check_library_version(request, python_cmd, "pygame")

All the code that was in this function is relevant to the task of checking the version of any library, so it all gets moved to the utility function. This one-line fixture function does still need to be in the test_alien_invasion.py module, however, because this is where we know which library to check for. Also, general utility functions can’t use fixtures as parameters. So, we have to keep the request and python_cmd fixtures here, and pass those values to the utility function.

Here’s the generalized utility function for checking and installing specific versions of a library:

def check_library_version(request, python_cmd, lib_name):
    """Install a specific version of a library if needed."""
    lib_version = request.config.getoption(f"--{lib_name}-version")

    if lib_version:
        print(f"\n*** Installing {lib_name} {lib_version}\n")
        cmd = f"{python_cmd} -m pip install {lib_name}=={lib_version}"
        output = run_command(cmd)
        print(output)

    # Regardless of what version was requested,
    # show which version is being used.
    cmd = f"{python_cmd} -m pip freeze | grep {lib_name}"
    result = subprocess.run(cmd, capture_output=True,
            text=True, check=True, shell=True)
    output = result.stdout.strip()

    print(f"\n*** Running tests with {output}\n")

This works for installing specific versions of Pygame, and it should work for the other libraries we’ll need to check as well. I’m printing the output of the pip install command, because it’s helpful to see that output while the test is running.

Issues with managing environments can be subtle, so regardless of which version was requested we display the actual version that’s being used for testing. We do this with the following command:

$ python -m pip freeze | grep pygame

The | character is a pipe; we’re piping the output of pip freeze to grep, which shows any lines that include “pygame”. When you include a pipe in a command that’s executed through subprocess.run(), you have to include the argument shell=True. Otherwise the output of the first command goes to a different shell environment. Also, the command needs to be submitted as a single string, rather than a list of parts. This means we can’t use the utils.run_command() function. There’s no need to adapt utils.run_command() to this usage, we’ll just call subprocess.run() directly here.

Now the output is clear about what version of the library is being used for testing:

(.venv)$ pytest tests/test_alien_invasion.py --pygame-version 2.4.0 -qs
*** Installing pygame 2.4.0
Collecting pygame==2.4.0
    ...
*** Running tests with pygame==2.4.0
pygame 2.4.0 (SDL 2.26.4, Python 3.11.5)
    ...
.
***** Tests were run with: Python 3.11.5
1 passed in 10.54s

We get a message about what’s being installed, and confirmation about exactly which version is being used. There’s a bit of redundant output here because things are going well, but it’s really helpful to have all this information available when things don’t go as expected.

Windows compatibility

I do most of my work on macOS, and forget sometimes that I’m using tools that aren’t available on Windows. The previous version doesn’t work on Windows because of the pipe to grep.

We can fix that by getting all of the output from pip freeze, and then filtering for the output specific to the library we’re focusing on:

def check_library_version(request, python_cmd, lib_name):
    """Install a specific version of a library if needed."""
    ...

    # Regardless of what version was requested,
    # show which version is being used.
    cmd = f"{python_cmd} -m pip freeze"
    output = run_command(cmd)
    lib_output = [
        line for line in output.splitlines()
        if lib_name in line
    ]

    print(f"\n*** Running tests with {lib_output}\n")

This works on both macOS and Windows, and I see no reason it won’t work on Linux as well.

Specifying requirements

At this point, we haven’t specified requirements for the test suite. The book’s repository doesn’t have a set of requirements, because the code isn’t usually run directly from the repository. It’s usually just used as a reference for the book.

Here are the requirements for the tests that have been written so far:

(.venv)$ pip freeze > requirements.txt
(.venv)$ cat requirements.txt 
iniconfig==2.0.0
packaging==23.2
pluggy==1.3.0
pygame==2.5.2
pytest==7.4.4

There are just a small set of requirements, but pinning them makes it easier to run the tests on different systems. It will also make it possible to reset the test environment to its original state, after installing specific versions of libraries during individual test runs.

Resetting the test environment

There’s one problem with the way we’ve implemented support for —-pygame-version. There’s a mismatch between the requirements.txt file that pins the versions for default test runs, and what’s actually installed after specifying a different version of Pygame. For example, here’s what I see after doing a test run with Pygame 2.4.0:

(.venv)$ cat requirements.txt| grep pygame
pygame==2.5.2
(.venv)$ pip freeze | grep pygame
pygame==2.4.0

The requirements specify Pygame 2.5.2, but what’s actually installed is whatever version the last test run used.

A simple way to address this is to reinstall from requirements.txt at the end of every test run. This is a really quick and straightforward way to make sure the environment returns to a consistent state after each test session.

We can do this cleanup work in the pytest_sessionfinish() function, in conftest.py:

def pytest_sessionfinish(session, exitstatus):
    """Custom cleanup work."""
    python_cmd = utils.get_python_cmd()

    # Reset any libraries that had a different version installed
    #   during the test run.
    print("\n\n--- Resetting test venv ---\n")

    req_txt_path = Path(__file__).parents[1] / "requirements.txt"
    req_txt_path = req_txt_path.as_posix()

    cmd = f"{python_cmd} -m pip install -r {req_txt_path}"
    output = utils.run_command(cmd)

    changed_lines = [
        line for line in output.split("\n")
        if "Requirement already satisfied" not in line
    ]
    
    if changed_lines:
        for line in changed_lines:
            print(line)
    else:
        print("  No packages were modified.")

    print("\n--- Finished resetting test venv ---\n")

    # Show which version of Python was used for tests.
    ...

We get the value for python_cmd at the start of the function, because it’s used to reset the virtual environment and to show which version of Python was used. We get the path to requirements.txt, and then call pip install. We filter for any lines related to a package that needed to be reset, and print that output. If every requirement was already satisfied, we print an indication that nothing was modified during the test run:

$ pytest tests/test_alien_invasion.py -q -s --pygame-version 2.4.0
*** Installing pygame 2.4.0
Collecting pygame==2.4.0
    ...
*** Running tests with ['pygame==2.4.0']
.
--- Resetting test venv ---
Collecting pygame==2.5.2
    ...
Successfully installed pygame-2.5.2
--- Finished resetting test venv ---
***** Tests were run with: Python 3.11.5
1 passed in 16.60s

This test still takes about 6 seconds if you’re using a version of Pygame that’s already been used recently. If a new version was installed, the test takes about 16 seconds. That’s not surprising, and is still so much faster than testing different versions manually.

I’ll note that this does not currently work for me on Windows. I think pip isn’t cleaning up its temp directories when it installs a specific version of Pygame, and those directories are interfering with the final call to pip install. I’m going to see if this is an issue with all libraries, or just Pygame, and sort it out a little later.

The default output that pytest shows is really useful for most testing scenarios. This extra output is quite helpful when testing against particular versions. If I request a specific version for testing and see the message No packages were modified, I know something went wrong with the test run. This is much better than a test run that just silently passes when it shouldn’t.

Running this test last

There’s one last modification I’d like to make to this test. The Pygame window disappears by the time the test suite finishes, but I can’t find a way to make it disappear as soon as the test for Alien Invasion finishes. A workaround is to make sure this test runs last.

The pytest-ordering plugin lets you control some aspects of the order in which tests are run. My needs are simple; I want this one test module to run last. After installing pytest-ordering, this one line at the top of test_alien_invasion.py causes the module to be run last:

pytestmark = pytest.mark.run(order=-1)

@pytest.fixture(scope="module", autouse=True)
def check_pygame_version(request, python_cmd):
    ...

Now when you run the entire test suite the Alien Invasion test runs last, and the Pygame window doesn’t hang around covering the terminal output.

Conclusions

Some projects can seem untestable, especially graphic-oriented projects. But if you can run your code, you can almost certainly test your code. Articulating the exact kind of tests you want to run, and what you want to get out of your test suite, can help determine how to approach building your test suite.

Standard tools like tox can be used to run test suites efficiently, in the ways most people need to test their software projects. If your needs don’t match the typical use cases of those tools, you can still manage your test environment to achieve the testing goals you originally set. This is part of why it’s good to articulate your goals before you start writing test code. Otherwise you can fall into the trap of writing whatever kinds of tests are most natural to write with the tools you have at hand, rather than the ones you actually need.

In testing, more information is often better than less information. It’s certainly worthwhile to have the ability to run tests with terse output, but you also probably want the option to get detailed information about a test run. Don’t be afraid to insert diagnostics into your test suite, that you can turn to when things aren’t behaving as you expect them to.

In the next post we’ll start to test the data visualization projects. This will bring up a number of issues related to making assertions about output in the form of images and HTML files.

Resources

You can find the code files from this post in the mp_testing_pcc_3e GitHub repository.

I’ve been hearing about Streamlit for a while now, and I’ve been looking for a project where it would be appropriate to use it. I recently found such a project, and Streamlit is just as impressive to work with as I’d heard. In this post I’ll go over just enough to help you get started building your own dashboards with Streamlit on your own system.

In a followup post, I’ll show how to quickly deploy your dashboards so others can use what you’ve created. I’ll also share the real-world project I’ve been using Streamlit to implement.

What is Streamlit?

In order to understand Streamlit, we need to be clear about what we mean by a dashboard. In the programming world, a dashboard is a way to view the most important data about a system. It’s usually a snapshot of a system where the data can be changing rapidly. Or, it can be a static dataset that you can tweak by adjusting the input values.

Streamlit is a tool for building dashboards using Python. You focus on the kind of data you want to present and what you want users to be able to do with it, and the library handles the presentation aspect of the dashboard. It’s an open source tool, and the company that developed it has a platform where you can host your open source projects. Because it’s open source, you can also self-host your own dashboards.

When using Streamlit, you don’t need to deal with HTML, CSS, or JavaScript at all. You just write .py files, and then run your main file with the streamlit command. Streamlit starts up a server, and shows your dashboard in a browser tab.

Rolling Dice

To explore Streamlit, let’s make an app that lets you roll some dice. Once we get it working, we’ll add a few features to see the kinds of things that Streamlit lets you do.

Let’s get started by installing Streamlit, and making a file to work in:

$ python -m venv .venv
$ source .venv/bin/activate
(.venv)$ pip install streamlit
...
Successfully installed MarkupSafe-2.1.4...streamlit-1.30.0...
(.venv)$ touch roller.py

The main streamlit library includes a whole bunch of packages for working with data, including pandas, numpy, and a number of well-known plotting libraries.

Now let’s write some code in roller.py that simulates rolling a single 6-sided die:

from random import randint
import streamlit as st

roll = randint(1, 6)

f"You rolled a {roll}."

It’s conventional to import streamlit with the alias st. We use randint() to get a random integer between 1 and 6. Notice the last line is just a string; there’s no print() call, or anything else. When Streamlit finds a bare piece of data on a line, it presents that data on the dashboard if at all possible.

You run this program with the streamlit run command:

(.venv)$ streamlit run roller.py 

      👋 Welcome to Streamlit!
      ...

  For better performance, install the Watchdog module:
  $ pip install watchdog

The first time you run streamlit, it will show a welcome message and ask you for your email. There’s also some telemetry practices you should read about. Briefly, they want to send some usage data to their servers when the streamlit server runs. You can write a two-line configuration file that prevents this telemetry from running:

[browser]
gatherUsageStats = false

The welcome message tells you where to save this file.

They also recommend you install watchdog, a package that helps libraries watch for changes in your source code files. I’ve installed watchdog to this virtual environment.

In addition to showing the welcome message, streamlit opens a new browser tab:

This is the start of our dashboard; it shows the message from the last line of roller.py. If you refresh the page, you’ll get a different number.

The st.write() function

Streamlit comes with a bunch of functions for writing data to the dashboard. The first to be aware of is st.write(). This function can write a wide range of data to the dashboard such as strings, markdown, LaTex expressions, DataFrames, Python objects, chart objects, and more.

This version of roller.py has the same output that we just saw:

...
roll = randint(1, 6)

st.write(f"You rolled a {roll}.")

The first time you change your main .py file, you’ll see a number of options at the top of the dashboard:

Streamlit noticed that your source code changed, and it wants to know what to do. You can rerun the file manually, or you can have Streamlit Always rerun the file when it notices a change. I’ve found this option to be useful so far in my own work.

Streamlit’s documentation is really easy to navigate. For a good place to start, check out the documentation page for st.write().

Input widgets: st.button()

It’s not very satisfying to have to reload the page in order to get a new roll. Let’s add a button that generates a new roll. Here’s how simple it is:

st.button("Roll")

roll = randint(1, 6)

st.write(f"You rolled a {roll}.")

This one line adds a button to the dashboard, with the label “Roll”. Streamlit re-runs your entire file every time the user interacts with an input element. Without attaching any code to this button, clicking it causes a new number to be rolled:

There’s a lot more you can do with buttons; check out the st.button() documentation page.

Input widgets: st.radio()

A dashboard with only one size die isn’t very useful, so let’s allow the user to choose what size die they’re rolling. We can do that with a radio select element:

side_options = [6, 10, 12, 20]
num_sides = st.radio("Number of sides:", side_options)

st.button("Roll")

roll = randint(1, num_sides)

st.write(f"You rolled a {roll}.")

This is where it starts to get interesting; we’re allowing the user to make choices that affect the values that appear on the dashboard. The side_options list defines what size dice the user can choose from. The st.radio() function generates a radio input element, and when the user selects a value it’s assigned to num_sides.

The call to randint() now uses num_sides as its upper limit:

I wouldn’t say this looks like a dashboard yet, but it’s starting to look more interactive.

Input widgets: st.slider()

In most situations where you want to roll dice, you don’t just roll a single die. Let’s allow the user to choose how many dice they want to roll:

# Input widgets
side_options = [6, 10, 12, 20]
num_sides = st.radio("Number of sides:", side_options)
num_dice = st.slider("Number of dice:", 1, 10, value=2)

st.button("Roll")

# Roll calculation
rolls = [randint(1, num_sides) for _ in range(num_dice)]
roll = sum(rolls)

# Output message
st.write("---")
st.subheader(roll)
st.write(str(rolls))

There’s a little more going on here, because we’ve got more dice to manage. But the streamlit-specific elements are fairly straightforward. We use st.slider() to let the user choose the number of dice to roll, from one to ten.

Rather than one call to randint(), we move the randint() call into a comprehension that generates the correct number of rolls. The roll variable now reflects the sum of these individual values.

For the output, we use the fact that Streamlit presents string values as markdown by default to place a divider between the input section of the dashboard, and the summary. We use the st.subheader() function to make the most important piece of information stand out, and we print a string representation of the full list of rolls:

If you call st.write(rolls) without converting rolls to a string, the same information will be presented but it will take up more vertical space.

Including a chart

What’s a dashboard without a chart? Let’s add a chart that shows the distribution of a large number of the kind of rolls the user is making. We’ll let them choose how many rolls to include in the simulation:

from random import randint

import altair as alt
import pandas as pd
import streamlit as st


# Input widgets
side_options = [6, 10, 12, 20]
num_sides = st.radio("Number of sides:", side_options)
num_dice = st.slider("Number of dice:", 1, 10, value=2)
num_rolls_sim = st.slider("Number of rolls in simulation",
        1_000, 100_000, value=1_000, step=1_000)

st.button("Roll")

# Roll calculation
rolls = [randint(1, num_sides) for _ in range(num_dice)]
roll = sum(rolls)

# Simulation rolls
sim_rolls = []
for _ in range(num_rolls_sim):
    sim_roll = sum(
        [randint(1, num_sides) for _ in range(num_dice)])
    sim_rolls.append(sim_roll)
df_sim = pd.DataFrame({"rolls": sim_rolls})

# Create histogram
chart = alt.Chart(df_sim).mark_bar().encode(
    alt.X("rolls", bin=True),
    y="count()",
)
chart.title = f"Simulation of {num_rolls_sim} rolls"

# Output
st.write("---")
st.subheader(roll)
st.write(str(rolls))

st.write("---")
st.altair_chart(chart)

You don’t need to follow this listing closely to get the main point. We import pandas to work with a DataFrame, and we import altair to make a chart. We add a second slider that lets the user choose a number from 1,000 to 100,000, in increments of 1,000.

We make a list called sim_rolls, and generate as many rolls as the user requested. We then create a DataFrame from those rolls. This DataFrame, df_sim, is passed to alt.Chart() to create a simple histogram.

The final two lines create another section, and write the chart to the page:

Okay, that’s an interesting number of elements to have on the dashboard. Let’s take a moment to reorganize them.

Structuring the dashboard

Streamlit offers a number of ways to organize your dashboard, without having to resort to writing HTML. For one thing, we can easily stick most of the input widgets in a sidebar:

...
import streamlit as st

# Sidebar

# Input widgets.
side_options = [6, 10, 12, 20]
num_sides = st.sidebar.radio("Number of sides:", side_options)
num_dice = st.sidebar.slider("Number of dice:", 1, 10, value=2)
num_rolls_sim = st.sidebar.slider("Number of rolls in simulation",
        1_000, 100_000, value=1_000, step=1_000)

# Roll calculation.
...

# Create histogram.
...

# Main page
st.title("Rolling Dice")
st.button("Roll!")

st.write("---")
...

By simply converting st.radio() to st.sidebar.radio(), widgets can be placed in a sidebar. This is an easy way to group some elements of your dashboard off to the side, and put more focus on the elements in the main part of the page.

I also added a title to the main part of the page, and moved the Roll button just below the title. These minor changes make a much more presentable dashboard:

That’s a pretty nice dashboard for about 40 lines of code!

Conclusions

Streamlit is even easier to get started with than I expected. By far, most of the complexity I’ve found centers around what I’m trying to do with the data; little complexity or difficulty has come from Streamlit itself. If you’re interested in building your own dashboard, take a look at the documentation of the elements used here, and see for yourself how clearly it’s laid out:

• Documentation home page.

• st.write()

• st.button()

• st.radio()

• st.slider()

• st.altair_chart()

• st.title()

• st.sidebar()

There’s more to cover, so I’m going to write at least two followup posts. The next one will show how to deploy the dashboard we just made using Streamlit’s public hosting platform. After that, I’ll show an example of a real-world dashboard built with Streamlit.

Resources

You can find the code files from this post in the mostly_python GitHub repository.

In the previous post, I laid out an overall plan for testing all the code in the book. In this post, we’ll put that plan into action. We’ll make a fork of the book’s repository, and build a test suite in the new repository. We’ll do this in a way that allows us to pull in changes from the original repository, without any conflicts with the testing code.

We’ll also use parametrization to write a set of tests that cover all the basic scripts in the first half of the book. These tests will run each script that should be tested, and make sure the output is what we expect it to be.

You can follow along and run the code yourself, if you’re interested in doing so.

Fork the original repository

Making a private fork of a public repository is not as straightforward as it sounds. GitHub doesn’t automate the process. I believe that’s partially for technical reasons, and partially to encourage public forks. The best resource I found was this gist, and this official post about forking.

I’m going to set up a public fork for this post, called mp_testing_pcc_3e. You can’t make a direct fork of your own repository; I ended up cloning the original pcc_3e repo, and pushing it to a new repo. If you want to follow along, you can simply make a fork of the original pcc_3e repository.

Testing basic programs

We’ll start by testing the basic programs in the book. These are programs that don’t use any third party libraries. They don’t write or modify any files, and they don’t accept user input. All we need to do is run them, and validate the output.

We’ll start by testing the first program in the book, hello_world.py. Just as a typical Hello World program demonstrates that your environment is set up correctly, this first test will show that our testing infrastructure is set up correctly and that a simple test can pass.1

Make a virtual environment, activate it, and install pytest:

$ python -m venv .venv
$ source .venv/bin/activate
(.venv)$ pip install --upgrade pip
...
Successfully installed pip-23.3.2
(.venv)$ pip install pytest
...
Successfully installed ... pytest-7.4.4

Make a tests/ directory, and in that directory make a new file called test_basic_programs.py. Here’s the first test:

import subprocess, sys
from pathlib import Path
from shlex import split

def test_basic_program():
    """Test a program that only prints output."""
    root_dir = Path(__file__).parents[1]
    path = root_dir / "chapter_01" / "hello_world.py"

    # Use the venv python.
    python_cmd = f"{sys.prefix}/bin/python"
    cmd = f"{python_cmd} {path}"

    # Run the command, and make assertions.
    cmd_parts = split(cmd)
    result = subprocess.run(cmd_parts,
        capture_output=True, text=True, check=True)
    output = result.stdout.strip()

    assert output == "Hello Python world!"

This isn’t the friendliest first test to see if you’re new to testing, but it does address the issue of many testing tutorials being overly simplistic. You don’t have to understand everything you see in this test function. Feel free to skim the following explanation of the test function, and then focus on making sense of everything that follows.

The test function has a generic name, because I’m planning to use this one function to run tests for all the basic programs. To build a path to hello_world.py we first look at the location of the current file, and then get the second element of the parents attribute. This should be the root directly of the overall repository. We can then define the path to hello_world.py.

To run hello_world.py for testing purposes, we want to use the Python interpreter from the active virtual environment. This will allow us to define a number of virtual environments, each with a different version of Python, and use any one of those to run tests. The command shown here doesn’t work on Windows, but we’ll fix that shortly.

The command we want to test is this one:

$ python hello_world.py

That command is built from the string f”{python_cmd} {path}”. We need to split this command into its parts before calling it with subprocess.run(). Finally, we assert that the output is what we expect it to be.

This test passes:

(.venv)$ pytest tests/test_basic_programs.py -q
. [100%]
1 passed in 0.01s

It’s really good to make sure your first test passes before writing more tests. This shows that the first test passes, but also that the testing infrastructure is set up correctly. Note that I’m using the -q flag here to save space; if you’re following along it’s more informative to omit that flag.

Also note that calling pytest without any arguments will fail at this point. If you try it, pytest will discover the example tests included in the book, and attempt to run them. We’ll address that issue in a moment.

Parametrizing tests

I know I want to run this same kind of test a bunch of times, for a bunch of different programs. If this first test runs using parametrization, we can add as many tests as we want, without modifying the actual test function.

Here’s the first test, parametrized:

import subprocess, sys
...

import pytest

basic_programs = [
    ("chapter_01/hello_world.py", "Hello Python world!"),
]

@pytest.mark.parametrize(
    "file_path, expected_output", basic_programs)
def test_basic_program(file_path, expected_output):
    """Test a program that only prints output."""
    root_dir = Path(__file__).parents[1]
    path = root_dir / file_path

    # Use the venv python.
    ...

    # Run the command, and make assertions.
    ...

    assert output == expected_output

To parametrize tests we make a list that stores the data that should be passed to the test function. In this case, I’m calling that list basic_programs. Each entry is a tuple containing the path to the file that needs to be tested, and the output that’s expected when that file is run.

The @pytest.mark.parametrize() decorator feeds data into a test function. You provide a name (or names) for the data, and a source of the data. Here we’re passing in the list basic_programs. pytest will feed one item at a time from that list into the test function. Right now basic_programs consists of one tuple, containing a file path and an output string. The items in this tuple are unpacked to the two names we provided: file_path, and expected_output. In this example, file_path will be the path to hello_world.py, and expected_output will be the string Hello Python world!

Inside the function, we only need to do two things: replace the hardcoded path to hello_world.py with the variable file_path, and replace the hardcoded output “Hello Python world!” with the variable expected_output. This test passes, with the same output as shown previously.

This can seem like a lot of complexity if you haven’t used parametrization before. But you’re about to see how much simpler it is to add more tests with this approach.

Can a test fail?

Before writing more tests, it’s a good idea to make sure your first test fails when you expect it to. I’m going to change the value of expected_output, and make sure the test fails:

basic_programs = [
    ("chapter_01/hello_world.py", "Goodbye Python world!"),
]

I changed Hello to Goodbye, ran the test, and it failed. That’s a good thing! Sometimes a test isn’t really running the way you think it is, and a change like this will still pass. If that happens, it’s much easier to troubleshoot now than later.

Make sure you undo the change that caused the failure, and run your test one more time to make sure it passes again. It’s so easy to introduce a bug that’s difficult to sort out if you don’t run these quick checks while building a test suite.2

Testing more basic programs

Now we get to see the benefit of parametrization. We have a test that should work for any basic Python program that simply prints output, as long as we give it the path to the program and the expected output. Let’s start by adding two more tests, and make sure they run (and pass):

basic_programs = [
    # Chapter 1
    ('chapter_01/hello_world.py', 'Hello Python world!'),

    # Chapter 2
    ('chapter_02/apostrophe.py', "One of...community."),
    ('chapter_02/comment.py', "Hello Python people!"),
]

These three tests pass:

$ pytest tests/test_basic_programs.py -q
... [100%]
3 passed in 0.03s

That’s very satisfying to see; we’ve tripled the number of tests, without adding any new test functions!

Testing all basic programs

At this point, creating more tests is as simple as adding new entries to the list basic_programs. I opened a terminal, cd’d into each chapter directory, and ran the programs I wanted to test.

I use GPT almost every day now, and this was a perfect example of why. Many of the basic programs in the book generate multiple lines of code. For programs like this, I wanted the multiline output compressed to single lines. For example this output:

Hello Python world!
Hello Python Crash Course world!

should be converted to:

Hello Python world!\nHello Python Crash Course world!

For two lines this isn’t a big deal, but for many programs with 5-10 lines that gets tedious. That’s the kind of thing AI assistants are great at, so I gave GPT a prompt telling it I’d be feeding in multiple lines, and asked it to convert those lines to the equivalent single-line Python string. I could write a program to do this, but it was much easier to just feed the output into GPT. It didn’t take too long to have tests for all 47 basic programs in the first half of the book:

(.venv)$ pytest tests/test_basic_programs.py -q
............................................... [100%]
47 passed in 0.49s

We’ve only written one test function, but we’ve got 47 tests now! This is a pretty good start. Before writing more test functions, let’s pull anything out of the existing test function that’s going to be used in other tests.

Getting python_cmd from a fixture

Almost every test is going to use the value for python_cmd that’s set in test_basic_program(). Let’s pull that out into a fixture, so it can be used by any test that needs it.3

You can place pytest configuration files in any directory in your test suite. Like a lot of things in the testing world this can be confusing at first, but it lets you structure an increasingly complex test suite in an organized way.

Make a file called conftest.py, in the tests/ directory:

import sys

import pytest

@pytest.fixture(scope="session")
def python_cmd():
    """Return path to the venv Python interpreter."""
    return f"{sys.prefix}/bin/python"

Fixtures are designed to make it easier to set up and share resources that are needed by more than one test function. Fixtures can be created once per session, module, class, or function.

Here we define a function called python_cmd(), that’s run once for the entire test session. The return value is the path to the Python interpreter for the current active virtual environment. Any test function can use this value by including the name python_cmd in its list of arguments.

Here’s what test_basic_program() should look like now:

@pytest.mark.parametrize(...)
def test_basic_program(python_cmd, file_path, expected_output):
    """Test a program that only prints output."""
    root_dir = Path(__file__).parents[1]
    path = root_dir / file_path

    # Run the command, and make assertions.
    cmd = f"{python_cmd} {path}"
    cmd_parts = split(cmd)
    ...

The main change here is the addition of the python_cmd argument in the definition of test_basic_program(). When pytest finds an argument name that matches the name of a fixture function, it runs that function and passes the return value to the argument.

In this case pytest sees the python_cmd argument, and recognizes that there’s a fixture function in conftest.py with that same name. It runs that function and assigns the return value, which is the path to the virtual environment’s Python interpreter, to the parameter python_cmd. Note that since the fixture function has a session scope, pytest only runs the function once. It will pass the return value to any function that has python_cmd as one of its arguments, without running the fixture function again.

In the body of the function, we got rid of the code that defined python_cmd, and reorganized the other lines slightly. All the tests still pass.

Pulling out some utils

Let’s also start a utils file, to gather code that doesn’t need to be a fixture, but will be used by multiple tests.

Make a file in the tests/ directory called utils.py. The first thing we’ll put there is a function to run a command:

from shlex import split
import subprocess

def run_command(cmd):
    """Run a command, and return the output."""
    cmd_parts = split(cmd)
    result = subprocess.run(cmd_parts,
        capture_output=True, text=True, check=True)
    
    return result.stdout.strip()

This function takes in a command, splits it into parts, and runs it by calling subprocess.run(). It returns a cleaned-up version of whatever was printed to stdout, which is where the output of print() calls is sent. This will make it easier to run a variety of programs , without having subprocess.run() calls in every test function.

Here are the changes to test_basic_programs.py:

...
import pytest

import utils

basic_programs = [
    ...
]

@pytest.mark.parametrize(...)
def test_basic_program(...):
    """Test a program that only prints output."""
    root_dir = Path(__file__).parents[1]
    path = root_dir / file_path

    # Run the command, and make assertions.
    cmd = f"{python_cmd} {path}"
    output = utils.run_command(cmd)

    assert output == expected_output

Here we’re importing the new utils module, and calling utils.run_command(). We’ve also removed the code that was replaced by the new utility function.

The tests all still pass, and we have a test function that’s much more readable than what we started with. More importantly, it will be easier to write more test functions that build on this one.

Ignoring example tests

I mentioned earlier that you can have multiple pytest configuration files, that control the behavior of different aspects of the test suite. We can simplify the command for running the test suite by ignoring the tests in the repository that are examples from the book.

In the root directory of the repository, make a new file called conftest.py. (Make sure you don’t overwrite the conftest.py file that’s already in the tests/ directory.)

In this new file, add the following:

"""Root pytest configuration file."""

# Ignore example tests from the book.
collect_ignore = [
    "chapter_07",
    "chapter_11",
    "solution_files",
]

When you have multiple conftest.py files, it’s helpful to add a module-level docstring that makes it clear which one you’re working with.

When you run pytest it scans your entire repository, looking for modules and functions associated with tests. This is called collecting tests. If pytest finds a list called collect_ignore in any conftest.py file, it will avoid collecting tests from the specified directories.

With this configuration file in place, we no longer need to tell pytest where the tests we want to run are. We can just use the bare pytest command:

(.venv)$ pytest -q
............................................... [100%]
47 passed in 0.58s

This is much nicer for running tests, and also less prone to typos.

Running on Windows

It’s a good time to make these tests work on Windows, because the only thing that should be different is the code that builds python_cmd. We should be able to add some conditional code to the python_cmd() fixture that returns the correct path on all systems. This is another benefit of having fixtures and utility functions; as the test suite grows in complexity, that complexity doesn’t overwhelm the overall structure of the test suite.

I’m approaching this by starting a Windows VM, cloning the test repository, and running the tests as they’re currently written. I’m expecting them to fail, and that’s exactly what happens:

(.venv)> pytest -q
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
47 failed in 4.64s

When you have a bunch of failures like this, it’s helpful to use the -x flag when calling pytest. This makes pytest stop after the first failed test, so you can focus on fixing one failure at a time. When most or all of the tests fail, fixing one failure often clears up most or all of the other failures.

Looking through the failure output closely, which I won’t reproduce here, shows that the issue is coming from the run_command() function. But the root issue is actually in the python_cmd() fixture function.

Virtual environments are structured slightly differently on Windows than on macOS and Linux. Instead of bin/python, the path to the Python interpreter on Windows is Scripts/python.exe.

Here’s the update to python_cmd(), in tests/conftest.py:

import sys
from pathlib import Path

import pytest

@pytest.fixture(scope="session")
def python_cmd():
    """Return the path to the venv Python interpreter."""
    if sys.platform == "win32":
        python_cmd = Path(sys.prefix) / "Scripts/python.exe"
    else:
        python_cmd = Path(sys.prefix) / "bin/python"

    return python_cmd.as_posix()

Strings don’t work well as paths across different OSes, which is why pathlib was developed. The as_posix() method returns a string that can be interpreted correctly on all systems in the way we’re using it, particularly in subprocess.run() calls. Here we define python_cmd as a path to the interpreter on each OS, and then return the as_posix() version of the appropriate path.

There’s a similar change that needs to be made to test_basic_program() as well:

@pytest.mark.parametrize(...)
def test_basic_program(...):
    ...

    # Run the command, and make assertions.
    cmd = f"{python_cmd} {path.as_posix()}"
    output = utils.run_command(cmd)

    assert output == expected_output

When we write commands, we need to use the posix version of paths. We’ve already done that for python_cmd, but we need to do that for the path to the file we’re running as well.

With these changes all 47 tests still pass on macOS, and now they pass on Windows as well. I don’t have any reason to think they won’t work on Linux at this point, although I haven’t tested that yet. It’s also worth noting that GPT is really helpful for sorting out cross-OS compatibility issues. You can run your code on a different OS, give GPT the entire error message, and ask it for suggestions. It doesn’t fix everything, but it frequently points out a promising direction for what to focus on.

Programs that need to be run from a specific directory

Some of the basic programs need to be run from a specific directory. These are all programs that read from a file, and use relative imports.

I added a new list of programs to test along with the corresponding output, and made a second test function that’s almost identical to test_basic_program():

import subprocess, sys, os
...

...
# Programs that must be run from their parent directory.
chdir_programs = [
    ("chapter_10/.../file_reader.py", "3.14..79"),
    ...
]

@pytest.mark.parametrize("file_path, expected_output", basic_programs)
def test_basic_program(python_cmd, file_path, expected_output):
    """Test a program that only prints output."""
    ...

@pytest.mark.parametrize("file_path, expected_output", chdir_programs)
def test_chdir_program(python_cmd, file_path, expected_output):
    """Test a program that must be run from the parent directory."""
    root_dir = Path(__file__).parents[1]
    path = root_dir / file_path

    # Change to the parent directory before running command.
    os.chdir(path.parent)

    # Run the command, and make assertions.
    cmd = f"{python_cmd} {path.as_posix()}"
    output = utils.run_command(cmd)

    assert output == expected_output

I called the new list chdir_programs, because these are basic programs where we must change directories before running them.

The new function is called test_chdir_program(), and the only difference between it and test_basic_program() is the call to os.chdir(). We change to the directory containing the file that’s being tested before running it, so the relative file paths work correctly.

At this point, we have 60 passing tests:

$ pytest -q
............................................................ [100%]
60 passed in 0.87s

When I first wrote the test suite, I placed some programs that had import statements in this group. It turns out those work without calling os.chdir(), so I added those programs to the set of basic tests in this commit.

You might have noticed the significant overlap between these two test functions. It’s not unusual to have more repetitive code in test suites. I know I won’t be adding more test functions to this file, so I’m going to leave these two test functions as they are, without any further refactoring. I also know the codebase isn’t going to grow beyond what’s already in the book, so there’s no reason to think any more tests will be added to this file. Don’t refactor blindly; if you have clear reasons to think you’ve reached a point where a particular body of code is good enough, be willing to call it good enough and move on to other things.

Conclusions

There are a number of takeaways from this first phase of building out the test suite:

• Start by writing a single test, which helps to make sure your testing infrastructure is working. Make sure your first test passes, but also make sure it’s capable of failing.

• Learn to parametrize your tests. It can seem complex if you haven’t done it before, but with a little work you can go from tens or hundreds of individual tests, to one or two tests that are used repeatedly.

• Pull common setup tasks out into fixtures.

• Pull common non-setup tasks out into utility functions.

If you address some or all of these points, you’ll probably have a test suite that makes it easier to carry out ongoing development and maintenance work. You’ll also have a test suite that’s easier to understand, and easier to build on as your project evolves.

In the next post we’ll make sure we can run the test suite using multiple versions of Python. We’ll also write tests for the first project, a 2d game called Alien Invasion that uses Pygame. Even if you’re not interested in games, it’s an interesting challenge to write automated tests for a project that seems to require user interactions. There are plenty of takeaways from this work that applies to a wide variety of projects as well.

Resources

You can find the code files from this post in the mp_testing_pcc_3e GitHub repository.

Maintaining a technical book is more challenging than many people realize. If you want your book to stay relevant, a technical book is never really finished. You have to keep an eye on the code to make sure it hasn’t gone out of date. When that inevitably happens, you need a plan for communicating updates to readers, and a plan for how to approach new editions.

Just as in more traditional software projects, automated testing can (and should) play a significant role in making sure a book’s code stays up to date. In this series I’ll share my current approach to testing the code for Python Crash Course. My workflow has evolved significantly over the course of three editions, and almost ten years now. There’s a lot of overlap between testing traditional software projects and testing code for a book, but there are interesting differences as well.

This series will have obvious relevance to current authors, and people who are considering writing a technical book. But even if you have no interest in writing, there are many elements of this work that are relevant to testing in general. For example, this series will touch on all of the following:

• How to plan a testing suite that meets your actual needs, rather than one that simply exercises as much code as possible;

• How to use parametrization to write a group of similar tests efficiently;

• How to test visual output such as plots;

• How to test programs that depend on user input;

• How to test programs that depend on random values;

• How to address cross-OS issues in testing;

• How to run tests in parallel, to speed up your test suite;

• How to use test artifacts to make debugging and ongoing development easier.

While the context centers around testing code from a book, the variety of challenges that come up will address issues of relevance to people interested in testing a wide variety of projects.

Series overview

For some series on Mostly Python, I draft each post as the series evolves. For others, like this one, I’ve planned out all the posts ahead of time. This series will consist of seven parts:

Part 1: How do you approach setting up a test suite? What kinds of things should you think about, and what decisions should you make, before writing any test code?

Part 2: How do you test a large set of basic Python scripts? How can you use parametrization to do this kind of testing efficiently? (MP #77)

Part 3: How do you test a game project? In general, how do you test a project that has visual output, and depends on user interactions?

Part 4: How do you test code that generates image-based data visualizations with Matplotlib? How do you validate static visual output?

Part 5: How do you test code that generates HTML-based visualizations, when the HTML is slightly different on every test run?

Part 6: How do you test a Django project, with a focus on external behaviors?

Part 7: What are the takeaways from all this testing work?

If you’ve been thinking about testing but haven’t written a test suite of your own yet, I think you’ll find a number of things in here that are helpful to your own work. If you have started to write tests, I believe you’ll find some aspects that look familiar, and some ways of doing things that help you think differently about your own testing routines.

A good testing mindset

I wrote in an earlier post that people should consider writing integration tests before unit tests, for untested projects and projects that are under rapid development. Chris Neugebauer gave a talk at PyCon Australia in 2015 with a similar message. There’s a great quote from that talk that has stayed with me:

If you have any code that can be run at all, there is a way to write tests for it.

A book is full of code. That code can be run, so there must be a way to write tests for it.

A brief history of testing Python Crash Course

When Python Crash Course first came out in 2015, I didn’t have any automated tests written for it. I posted all the code for the book online, and when readers pointed out inevitable mistakes I updated the repository, and made notes to fix those mistakes in the book the next time it went out for a new print run. With this approach, the book steadily got better over time.

Python Crash Course is really two books in one. The first half of the book is an introduction to the basics of Python and the fundamentals of programming. The second half of the book is a walk-through of three different projects: a video game, a series of data visualizations, and a web app that goes all the way through deployment.

This means the book’s longevity depends not only on how Python evolves, but also on the ongoing development of third party libraries for writing games, data visualizations, and web apps. Testing any one of these isn’t always straightforward; testing all of these in one comprehensive test suite is an interesting challenge.

Most of my early testing work for the book was manual. When a new version of Python was released, or a new version of any of the libraries the book depends on was released, I’d manually set up a virtual environment and run the code for each of the projects in the book. This worked, but it wasn’t particularly efficient. There were definitely a few times I didn’t do this work ahead of time, and just crossed my fingers that a new version of a library wouldn’t break the book’s code.

For the second edition of the book, which came out in 2019, I tried to write a proper test suite for the book. I wrote code that walked the entire repository for the book’s code, and tried to run all the files in the repository. That sort of worked, but it was a clunky mess of trying to exclude files that didn’t really need to be tested, or weren't testable in a straightforward way.

The third edition came out in January of 2023, and I’ve learned a lot more about testing since the previous edition came out. This time I laid out the goals I want to achieve with a test suite, and set out to build a set of tests that would do exactly that. Instead of walking through the repository and trying to test every file, I wrote a test suite that specifies exactly which files should be tested, and how. I’m really pleased with the result.

An ideal test suite

An ideal test suite would let me do the following:

Run all the most significant programs in the book. We often hear about the ideal of 100% test coverage, but that’s just not necessary with a book. For example I have folders called partial_programs/, which show all the versions of each file as it’s being developed in the text of the book. If the final version of each of these programs works, I’m confident enough in all the partial versions.

Run the test suite against different versions of Python, including release candidates. This is really important; I want to know that the code for the book will work on the latest version of Python. I want to test against release candidates of new versions, rather than waiting for the full public release of each new version.

Run the test suite against any version of any library used in the book. This is critical as well. The book covers pytest, Pygame, Matplotlib, Plotly, Requests, and Django. All these libraries have different lifecycles, and I need an easy way to test the projects in the book against any version of these libraries, including release candidates.

Drop into an active environment for a specific project. When maintaining the book, it gets quite tedious to set up an environment for a project in order to run it and look at certain aspects of that project. An ideal test suite will let me run the tests, and then drop into an active environment for any project in the book. This goes beyond testing; it’s testing that leaves artifacts behind that I can easily work with in a variety of ways. This is especially helpful when readers report possible issues, and the tests don’t necessarily cover the issue, but they do set up an environment where it would be easy to check out that issue.

Runs on macOS, Windows, and Linux. Readers use all these platforms, so I need to be able to test on each of these platforms as well.

Ideal test commands

It can be helpful to imagine the kinds of commands you’d like to be able to run if your ideal test suite already existed. These are the kinds of commands I’d like to be able to run for this project:

$ pytest tests/
$ pytest tests/test_django_project.py
$ pytest tests/test_django_project.py --django-version 5.0a1

Having a sense of what kinds of commands we want to run points us toward the testing files we should start to write, and the kinds of features we should be looking to build into the test suite. Here I want an overall tests/ directory, and I want individual test files for each of the major projects in the book. I also want to be able to include CLI arguments that let me specify exactly which version of a library to use for any given test run.

The overall plan

In most software projects the test suite lives alongside the project’s code, but that doesn’t quite work for this kind of testing. The main repository for the book is really for readers; trying to include a full author-focused testing suite in that repository is not ideal. I also don’t want to maintain these tests in public. They sometimes highlight issues that I recognize as minor, that might look like more significant breakage to people who don’t have the same perspective on the code and the purpose it serves.

When making an overall plan for the test suite, I also have to wrestle with one aspect of testing code for a book that’s different from testing standard projects. In a standard project, you’re somewhat free to change the code to make it more testable; this is one of the benefits of starting a test suite early on in a project. But a book’s code is “frozen”. It’s printed in physical books, and it’s developed with a higher priority on pedagogical goals than on performance or testability. So we need to come up with a way of testing that deals with all the code as it is. This isn’t entirely unique to testing code from a book. You may need to write tests for a project that includes code that you or your team don’t have direct control over.

Here’s the approach I came up with:

• Make a private fork of the book’s public repository.

• Add any files I want to this repository, to support testing.

• Don’t remove or modify any file that’s in the public repository. If I need to change something for testing purposes, make a copy of that resource and modify it during the test run.

• When I make changes in the book’s code for new printings, pull those changes into this repository as well.

• Never push anything from the test repository to the upstream public repository.

For this series, I’m going to make a public version of the test repository, which I’ll then archive as a demo when the series is complete.

Conclusions

Writing a test suite for the code in a technical book presents a unique challenge, especially if the book covers a variety of different topics, using a number of third-party libraries. However, addressing these unique challenges brings up ideas that are useful to people testing a wide range of projects.

If you’re developing a test suite for a project of your own, take a moment to step back and ask yourself what your larger goals are for the test suite. Don’t fall into the trap of just writing a bunch of tests that exercise most of your code. What do you want to get out of your test runs? What do you want to be able to do immediately after a test runs? How do you want to use your tests as a part of your development workflow? Tests don’t have to be something that’s only dealt with after you write your code; they can be something that are useful while actively developing your code.

In the next post we’ll write a set of tests that cover all the basic programs in the book. We’ll go through the process of writing the first test, making sure it can fail and then making sure it passes. We’ll use parametrization to quickly generate a large set of similar tests. We’ll also start to use fixtures to implement some features of the test suite that will be used throughout all the test modules.

Note: If you have questions or thoughts to share about testing, please add a comment below, or feel free to reply to this email and I’ll be happy to respond.

I bought a rowing machine a couple years ago, to help stay in shape when I didn’t have enough time to get outside for a longer run. I rowed crew in college, and that background makes it easy to hop on and do a quick, effective workout.

This fall I tried Concept 2’s Holiday Challenge, where you try to row 100 kilometers between Thanksgiving and Christmas. That’s about twice as much as I normally row. I just finished the challenge last week, and it had me thinking about what I’ve learned from attempting various streaks over the years.

Streaks are much more flexible and attainable than typical New Year’s resolutions. A resolution is supposed to be a permanent change, which is notoriously difficult to see through. A streak requires commitment, but it also has an end. If you choose the right kind of streak, the benefits can be much more attainable and significant.

Rowing 100k

Most of the time, I like to stay in shape by doing a number of different things: running, hiking, biking, paddling, and anything else I can do outdoors. But falls and winters in southeast Alaska are cold and rainy, and this winter is one of the wettest I’ve seen in 20 years here. It was a perfect fall to focus on indoor exercise.

When I row, I typically go for a specific time rather than a set distance. I’ll do 20 minutes at a faster pace, or 30-40 minutes at a slower pace. Before I started the 100k challenge I sketched out a rough schedule with a short, medium, and two longer rows each week. I did that for the first week, but then found I was really enjoying the longer rows. The core workout I settled on was a 10k row, which takes me just under 50 minutes at a steady pace.

Once I got into a routine, I enjoyed the longer rows so much I started pushing past that 10k mental limit. I did a 12k, and then a 15k row. I’ve always been much more of a distance athlete than a sprinter. I often find my pace around 30-40 minutes into a workout, and if I can make the time for it I really enjoy stretching that out to 60-90 minutes.

Physical benefits

Whenever I attempt a streak, there are some things I know I’ll get out of it if I’m successful. But I also know there will be some unexpected benefits as well, and I’m always curious to see what those end up being.

For this streak, I knew I’d get in better shape for rowing. That definitely happened, but I had no idea what directions it would take me. I’ve been surprised to find out how much I enjoy rowing longer distances. I rowed an 18k piece after the challenge was over, which worked out to just under 90 minutes. I chose that distance to see if I’m ready for a half-marathon, where you row the equivalent of a half-marathon in running (21,097 meters).

The 18k piece took more out of me than I expected, but I’m still planning to do the half marathon in the next week or so. We’re moving to North Carolina this summer, so much of our spring will be taken up with cleaning out from 20 years of living in one place, and sorting out the logistics of a cross-country move. If we weren’t taking that on, I’d keep going and prepare for a full marathon. I don’t want to spend too much of my life sitting on a rowing machine, but I’d love the satisfaction of rowing a marathon once.

I’ve also found that shorter pieces feel much easier than they used to. My results in 20-minute pieces are much better than they were before, because a 20-minute piece feels so much shorter and easier now. I can push myself harder for these shorter times, knowing it’s going to be over soon. I’m looking forward to testing myself on a variety of shorter pieces after finishing the half marathon.

Mental benefits

One of the things I really like about these longer rows is that they’re a perfect way to sit and watch the conference talks I’ve been wanting to see. For each long row, I set up a queue of talk videos. Throughout the month I watched about ten talks: all the talks I missed in person at DjangoCon US, the PyCon 2023 talks I wanted to see, and a few classic talks I’ve had on my playlist for a long time. After most workouts I’d write a few notes about how to use what I learned from these talks in my current projects.

For the longer rows I found it helpful to queue up technical talks for most of the workout, followed by outdoor videos for the final stretches. I couldn’t really concentrate on a technical talk when I was running out of energy and trying to push myself at the end of a long piece. I started putting climbing videos at the end of these playlists, and it was much easier to push myself while watching others push themselves. For anyone interested in long-distance hiking, a short documentary about the Pacific Crest Trail was really satisfying to watch during the end of a long row.

Aside: Concept2 is an amazing company

It’s worth sharing a little about the company that makes the rowing machines that almost all experienced rowers use. Concept 2 started as a small company that built rowing machines out of repurposed bicycle parts. Their machines have evolved steadily over the last 40+ years, but throughout this entire time they’ve focused on durability and backwards compatibility.

Concept 2 rowing machines (ergs, short for ergometers), are used by college and Olympic rowers every day for years on end. If you buy one for home use, it’s likely to last 20 years or more. And if it stops working, you can still buy parts for your 20-year-old machine. As a software developer who pays attention to backwards compatibility, this is an amazing thing to see from a company that focuses on athletic equipment.1

Other streaks

90 days of open source contributions

A while back, I completed a streak of making at least one commit to an open source project each day, for 90 days. That was in the early days of GitHub, when many people were seeing their contributions laid out visually for the first time:

That streak was interesting in that I didn’t set out to make it happen. I noticed one day that I had made at least one commit every day for the last 17 days. I decided to keep making daily contributions, without a specific end goal. I kept the streak alive as long as it was beneficial: if maintaining the streak helped me keep working when I was discouraged, or when the work wasn’t easy, I kept it going. I decided to end the streak when it started to make me not enjoy programming.

There were lots of takeaways from that period. Some people were trying year-long streaks at the time. Ninety days was enough to push myself professionally, but also to maintain a healthy work-life balance. It made me much more comfortable with Git workflows, such as the logistics of working on a variety of branches, and different ways of submitting PRs. This was also when I first started writing tests for my projects.2

A year of writing weekly posts

I started this newsletter one year ago. One of my reasons for starting it was to commit to writing on a more consistent basis. I’ve managed to write at least one post a week for a full year, averaging about six posts each month. While it’s been a lot of work, I’ve also grown a lot from the experience:

I can write more efficiently. I’ve been a writer for a long time now, so my overall writing process is pretty established. But one weakness I had for a long time was the need to write first drafts on paper. I just couldn’t put my thoughts together coherently when staring at a screen. I think that comes from doing most of my early writing on paper, and in journals. My first newsletter posts were drafted on paper, but I quickly transitioned to an almost entirely digital workflow. Now I tend to only draft on paper when I can’t seem to get into a flow with a screen in front of me.

I’ve dug into a wider variety of topics than I otherwise would have. Many people have the impression that technical authors already know everything they’re writing about. That’s rarely true. Often in my programming work, I learn enough about any given topic to solve the problem I’m currently facing. But when I’m writing, I want to make sure I’ve explained each topic correctly and accurately. Writing pushes me into areas I wouldn’t otherwise explore. It also pushes me to not just make things work, but make sure I understand how they work.

I’ve met a number of people I wouldn’t otherwise have met. My posts don’t tend to get a lot of comments, but people do write privately to share their reactions. Some of those exchanges have led to video calls and ongoing collaboration. I’m always deeply appreciative of the people that programming has brought me into contact with, all over the world. I recently joined a technical writers’ group, and those conversations, both formal and informal, have been fantastic.

I really enjoy writing about programming, and the projects I’m working on. I don’t do it for the streak, but the long-term impacts of this kind of writing reminds me of what comes out of a streak.

Conclusions

I find streaks much more beneficial than resolutions. A resolution tends to be something that we’re supposed to change permanently, which can be really hard to do. A streak represents a commitment, but it has an end which you can keep in sight. When we finish, we get to reflect on how we’ve grown, and choose how to build on the experience.

After finishing a meaningful streak, what was difficult at the start becomes easy; what seemed impossible is now within reach. I’ve found this true with physical challenges like rowing 100k, and mental challenges such as keeping up an open source contribution streak.

If you’re considering taking on a streak, keep a couple things in mind. Make sure to define one that’s attainable, but will push you toward outcomes you can’t fully know ahead of time. Choose something you want to get better at: programming, exercise, music, reading, etc. Choose a frequency that fits into your life: every day, once a week, once a month. Finally, choose an end goal that feels like a stretch, but also feels attainable.

When you reach your end goal, be wary of continuing the streak. It can be hard to let go, but streaks can easily lead to burnout if you focus too much on the streak itself. Instead, consider what aspects of the streak were enjoyable and worth keeping in your life, and let go of the aspects that turned into obligations you started to dread.

I wish you a good start to 2024, and if you’ve got a resolution or a streak in mind, I hope it goes well. :)

1

If you’re curious to learn more about Concept 2’s history, see their 40th anniversary page. Their first machines used bicycle wheels with plastic cards attached to the spokes for resistance. This was called the Model A:

The first machine that looks a lot like the current one was the Model B, released in 1986. There have been just a few major redesigns; the Model C was introduced in 1993, and the Model D was introduced in 2003. There are a couple more variations, but most of the evolution took place across these four models. You can still buy parts for all models, including the Model A. You can also buy retrofit kits to integrate the most important updates, such as the performance monitor, onto older machines. They truly don’t push people to replace machines that are still working.

Concept 2’s latest machines are called RowErgs, to differentiate from the bike and ski machines they also produce now. Their machines are also quite reasonably priced. They’re more expensive than the cheapest machines you can find, but they’re also much more affordable than the latest tablet-based machines. And there’s no subscriptions whatsoever.

Recently I was working on a section of code that uses a number of list comprehensions. I rarely use single-letter variable names, and was surprised to find a number of places where single-letter names made the code more readable.1

Allowed modifications

The code I’m working on modifies the user’s project so it’s ready for deployment to a remote host. Here’s the main form of the command that people run when using this project:

$ python manage.py simple_deploy --platform <platform-name>

This command makes configuration changes to the user’s project based on the platform name they specify. Users can then run their platform’s deploy or push command, and they should have a working deployment of their project.

Ideally, the user should have a clean git status before running this code; all the changes made to their project should be contained in a single commit. This makes it easy to see what configuration changes were required for the target platform. It also allows the user to easily roll their project back to a clean state if they decide they don’t like any of the changes that were made.

When users run the simple_deploy command, it runs git status in the background before making any changes. If the output indicates the presence of uncommitted changes, the project exits with a message asking the user to commit their existing changes and then run the command again.

However, there are some uncommitted changes that are acceptable, which shouldn’t block the code’s execution. For example, the user may have added django-simple-deploy to their project’s requirements. Or, they may have run the command once and fixed an issue that blocked configuration. Running the command creates a log directory, and that directory is added to .gitignore. We don’t want to block execution based on these kinds of changes.

Examining changed files

One of the simplest ways to check for uncommitted changes is to see which files have been changed. Here’s two lines of code from a function that checks whether it’s okay to proceed with modifying the user’s project:

if any([path.name not in allowed_modifications for path in modified_paths]):
    return False

This code looks for any file that’s been modified, that’s unrelated to a simple_deploy run. If any such files exist the function returns False, indicating it’s not okay to proceed.

Significant and insignificant names

Let’s look at just the list comprehension in this code:

[path.name not in allowed_modifications for path in modified_paths]

We’re thinking about how to name things, so let’s write down all the names used here:

path
allowed_modifications
path
modified_paths

Two of these names are defined outside the comprehension: allowed_modifications, and modified_paths. The other name, path, is only used inside the comprehension.2

Here’s a version of the comprehension that de-emphasizes the name path:

[p.name not in allowed_modifications for p in modified_paths]

We don’t often use single-letter variable names, because they lack context. But in a comprehension, all the context is contained in a single line. Using the name p here emphasizes a few things:

• It’s the name attribute of the path that we’re focusing on;

• We’re looking for names in allowed_modifications;

• The paths we’re examining are coming from modified_paths.

These are exactly the things that I want to call the reader’s attention to, if they’re unfamiliar with this codebase.

This is especially noticeable if we make the opposite kind of change, to a more verbose set of names:

[modified_path.name not in allowed_modifications for modified_path in modified_paths]

This is a common way to name things if we’re accustomed to using plural names for lists, and then using the singular version of that name in the opening line of a for loop:

for modified_path in modified_paths:
    ...

In the context of a full loop, where the first line is less busy than a comprehension, this naming approach works. That’s especially true if the block that follows has any degree of complexity.

Coming back to any()

If it’s not clear what this code does, consider the emphasis shown in this version of the comprehension:

[p.name not in allowed_modifications for p in modified_paths]

The bold expression here will always evaluate to True or False. So we’ll end up with a list like this:

[False, False, True, False]

Wrapping any() around this list returns True if any of the values in the list are True, and False otherwise:

>>> any([False, False, True, False])
True

The original code is a little hard to reason about out of context. If any of the modified files are not in the list of allowed modifications, the function returns False, indicating it’s not okay to proceed in configuring the user’s project. If none of the modified files are in that list, it will return True and we can proceed.

Conclusions

Naming things really is hard. I think when people say that, we often think about times where it was hard to come up with a descriptive name for an abstract concept we were working with. But many times there are smaller naming decisions that affect how readable our code is, especially to people who aren’t very familiar with the overall codebase.

When you’re writing a comprehension, consider using short names for the variables that only exist inside the comprehension. They should make sense to people reading your code, and draw attention to the more significant names that exist outside the comprehension itself. If you recognize other situations where a variable is only used in a single line, or in an otherwise isolated context, consider using short names there as well.

Don’t go overboard. For example, single-letter variable names in a standard for loop will probably make your code less readable.

This tension between wanting to let go of “work” and focus fully on holiday festivities has me thinking about all the projects I’ve done over the holidays in my life. Working on my own projects feels really good during the holidays, and brings up surprisingly strong feelings.

For most of my life, programming was not part of my daily work. I was a math and science teacher for a long time. Outside of a few school-related projects, I never got to write code during work hours. All my programming work had to be done in the early morning hours, late at night, or on weekends.

During those years, I did some of my best programming work over the holidays. A couple weeks of no school, the short days of winter, and fewer obligations meant I could curl up on the couch with a laptop and build out another side project. I remember one break when I wrote a C project to plot 3d functions, before plotting libraries were common. On another break I learned 3d modeling using OpenSCAD. I wrote a Python program that generated OpenSCAD code, and made a Koch snowflake. I had it printed in stainless steel, and now that ornament hangs on our tree every year.

I got my first mechanical keyboard as a Christmas gift one year. Everyone in our family gets to play with their gifts on Christmas, so I sat alone for a couple hours and loudly coded a new website.1 It was a tool for building math problems for my students based on what they were interested in, and I ended up using it throughout my last few years of teaching.

These are sweet memories, and I sometimes justify this time by pointing out how much I learned during those work sessions. But I wasn’t just working; programming is really fun and satisfying when you get to work on whatever project you want to.

If you’re writing code over the holidays, please know that you’re not alone. There’s a lot of people out there writing code for the simple joy it brings. There’s also a lot of people writing code because they’re at a point in life where you grab every spare minute you can, to learn as much as you can and hopefully build a better life for yourself.

Whatever your reason for writing code today and in the days that follow, I hope it’s going well. And if you’re taking a break from coding over the holidays, I hope you find renewed joy in your work when you get back to it. :)

I went to DjangoCon US 2023 in Durham this fall, and really enjoyed the talks I was able to attend in person. But there were a bunch more I wanted to see, and now that the recordings are out I’ve been working my way through the playlist.

One of the first talks I wanted to watch was Empathetic testing, by Marc Gibbons. It really resonated with my thoughts about testing, so I wanted to share some of his perspectives, and offer some reactions to the points he made.

What is a unit, anyway?

If you’ve dealt with testing at all, you almost certainly started with unit testing. Before reading further, take a moment and ask yourself what a unit is. Try to commit to a specific answer before reading on.

Despite what many people probably think, the term unit in unit testing doesn’t have a concrete definition that everyone agrees on. I want to keep this post short, so I’m just going to share my own working definition:

A unit is a block of code that’s worth testing in isolation.

Notice the phrasing here, block of code. I would guess that many people would define a unit as a function. But I’ve seen plenty of functions that are clearly larger than what I think of as a unit.1

Signs of a broken test suite

I enjoy watching tech talks because of all the real-world stories people share to put their bigger points in context. In this talk, Marc tells a story of a really broken test suite. The context was software for a financial firm, and the example he focused on was a section of code that calculated the amortization table for paying off a loan.2

Marc’s team wanted to start using NumPy for some of their financial calculations. After some refactoring, the existing tests all passed. However, the site was broken for end users. They fixed the code, so the site worked again for end users. But now the tests failed. This is exactly the opposite of how tests should work! The test were passing when the site was broken, but failed when the site was working.

It’s important to note that this was not a terrible test suite. The test suite had served its purpose for some time before this refactoring work. An imperfect test suite that’s run regularly is much better than a perfect test suite that hasn’t been written yet.

Avoid implementation bias

When Marc was discussing the broken test suite, he described it like this:

Our tests aren’t actually testing how things work; they’re testing the way the thing was built.

While I’ve been aware of this kind of issue with tests, I hadn’t heard it named before. Marc’s larger point was to avoid implementation bias. A unit test should verify the behavior of a unit. It’s rarely important, or even valuable, to verify the implementation of a unit. Yet it’s easy to fall into that approach to writing tests sometimes if you’re not aware of the problems it can create.

The test suite he was working on broke because some of the tests were verifying the behavior of units, in a way that was overly dependent on the implementation of those units. A brittle test is technical debt. It’s helpful only as long as the current implementation is in place. A test written with implementation bias hinders refactoring, rather than supporting the evolution of the codebase.

Empathetic testing

Marc had an interesting take on how to address this kind of brittleness in tests. Rather than just directing people to test behaviors in a way that’s independent of the implementation, he suggested that people should approach testing from an empathetic perspective. He encouraged people to think of the future maintainers of the projects we work on, not just from the perspective of imagining them running our test suite, but from the perspective of realizing that they’ll probably be able to write better code than we’re currently writing.

Some people laugh at these kinds of suggestions. They might say things like, “It’s just code, there’s no feelings here!” But programming is full of feelings. We have to feel a bit of ego to write working code in the first place.

Every time we sit down at a keyboard, there’s some part of us that says, “I am the one who can solve this problem!” We need some pride and hubris in order to start working on a project. Marc’s point was that another voice should remind us, “Someone may well come along and improve this implementation at some point.” Our tests should server ourselves when we’re writing them, but they should also serve those future developers who see the problem even more clearly than we do. (And sometimes, we are that future contributor.)

Conclusions

I have a number of takeaways from watching this talk, and reflecting on the points that were brought up:

• Testing is almost always about verifying behaviors, not implementations.

• We need to be thoughtful about the tests we write, even when writing small unit tests. We can ask ourselves, will this test support refactoring, or hinder refactoring?

• Think clearly about the “API” of your smallest pieces of code. If a function’s external usage is consistent and stable, we can write unit tests that interact with the function in a stable and non-brittle way. This holds true even as the code inside the function gets refactored.

Testing is not separate from programming. Writing, using, and understanding a test suite will help you understand your code better, and think more carefully about the overall structure of your project as well.

• Where to go to learn about the newest features in Django;

• How to make sense of Django’s version numbering system;

• How to see if your project can easily be upgraded to 5.0;

• How to deploy a Django project in just three steps.

If you’re using Django and aren’t already familiar with its release cycle and upgrade process, read on for a little insight into how it works.

What’s new in 5.0

A lot has already been written about the new features in Django 5.0. Rather than repeating what others have written, I’ll simply point you to some of the resources that I’ve found most helpful:

Official announcement

The official announcement is always short and to the point. It mentions a few highlights, and includes links to relevant parts of the documentation.

Release notes

If you want to keep up with Django development, it’s well worth your time to scroll through the entire set of release notes. I don’t encourage you to do a close reading of the entire document. However, becoming familiar with the structure of this document, and glancing through everything in it, will give you a good sense of how much active development Django sees in a single release cycle.

The most significant new features are mentioned in the official announcement, and in many blog posts. They’re also described in the first few sections of the release notes. But there’s a long list of minor features that are worth reading through, especially in areas of Django that are most relevant to your work. For example, if you do a lot of GIS work, you’d probably be interested in looking at the specific release notes for django.contrib.gis.

There are also sections about backwards incompatible changes, deprecated features, and features that were finally removed in 5.0. If you want to make sure your Django projects can be upgraded reasonably efficiently, paying attention to these deprecation sections on an ongoing basis is important.

New goodies in Django 5.0

Marius Felisiak is one of the Django Fellows. It’s his job to keep up with new developments in Django, so he’s in an excellent place to write up the new features that just came out.

Understanding Django’s version numbering system

Django uses a very consistent version numbering system, but unfortunately it’s a little misleading about what kind of changes people should expect to see in a new release.

I think people tend to associate X.0 releases of any library with a bunch of backwards-incompatible, breaking changes. While Django 5.0 does have some backwards-incompatible changes, they aren’t particularly significant. Each release of Django has some breaking changes; the goal is to manage these carefully, so that people can steadily upgrade their projects over long periods of time. If you read each version’s release notes, you’ll have over a year of notice about any upcoming deprecations that are relevant to your work.

Django’s numbering system

Django makes three releases per major version number: X.0, X.1, and X.2. A new version comes out about every eight months.

For example, these are the most recent releases:

4.0, 4.1, 4.2, 5.0

And here are the next few versions we’ll see:

5.1, 5.2, 6.0, 6.1, 6.2

LTS releases

It’s helpful to focus on the X.2 releases when trying to understand Django’s version numbering system. The X.2 releases are considered LTS (long-term support) releases.

Here’s how this is described in the 4.2 release notes:

Django 4.2 is designated as a long-term support release. It will receive security updates for at least three years after its release.

Non-LTS releases

The X.0 and X.1 releases receive active updates for eight months, and security updates for another eight months. This means Django 5.0 will receive active updates through August 2024, and security updates through April 2025.

Visualizing support

There’s a great visualization of the support lifetimes for each version in the Supported Versions section of Django’s Download page:

In a followup conversation after this post was initially released, I was reminded that there’s a better version of this timeline:

This visualization uses colors to indicate which state each release is currently in. Red indicates an older, unmaintained release. Green releases are receiving security updates. Yellow releases are receiving bug fixes and security updates, and blue versions are prereleases.

Recommendations

So which version should you use, and how should you think about upgrading? Django has built a well-deserved reputation for stability and reliability. If a feature makes it into a release, it’s almost certainly not going to cause you problems in production.

The old recommendation was to consider LTS releases unless you needed the features in a non-LTS release. That advice is outdated at this point. I would recommend the following:

• If you can commit to upgrading your project at least once a year, always upgrade to the most recent version that’s been released.

• If you can’t commit to upgrading at least once a year, consider focusing on LTS releases.

One advantage of staying on the latest release is that your upgrades are likely to be easier. Regardless of which strategy you adopt, you should always update your version to the latest point release. That is, if you’re using Django 5.0, you should upgrade to 5.0.1 as soon as it’s released. These minor point releases contain important security patches, and should never cause breaking changes.

Note: If you want to read all the details to inform your own upgrade policy, see the official documentation page for Django’s release process.

Upgrading a project

I have a project that’s currently running on Django 4.1. This is a sample project, but I’m going to show how to update it to run on Django 5.0. If you have a project that isn’t overly complex, you can probably follow a similar upgrade process.

Upgrading to the latest version is often not as hard as people think. You can try to upgrade in an isolated environment, and get a quick sense of how easily your upgrade path will be. This is especially true if you have a reasonably effective set of tests for your project.

BlogMaker Lite, on Django 4.1

BlogMaker Lite is a sample project I made a while back when developing a separate project focused on deployment. This is a relatively simple Django project, but it’s not trivial. It has user accounts, and uses a third-party package for styling.

I haven’t touched this project in a while. Let’s clone it, start the development server, and make sure it still works:1

$ git clone https://github.com/ehmatthes/dsd_sample_blog_reqtxt.git
$ cd dsd_sample_blog_reqtxt
$ python -m venv b_env
$ source b_env/bin/activate
(b_env)$ pip install -r requirements.txt
(b_env)$ python manage.py migrate
(b_env)$ python manage.py runserver

Here’s the project’s home page:

The home page renders fine.

Let’s run the tests, and see if the project functions correctly. This project doesn’t have a standard Django test suite. I was focused on using this project to run repeated deployment attempts, so I was most interested in whether the project functioned correctly regardless of where and how it was deployed. It’s a simple set of calls to the project’s URLs using requests, followed by assertions about the contents of the response object. These days I’d probably use pytest for this, but I have no pressing need to change the current test script.

Here’s the test run:

(b_env)$ python test_deployed_app_functionality.py \
    --url http://localhost:8000

Testing functionality of deployed app at http://localhost:8000/...

  Checking anonymous home page...
  Checking empty anonmyous all_blogs page...
  Checking empty anonmyous latest_posts page...
  ...

The tests all pass, including the ones that make a new user, a new blog, and a couple blog posts. The last test fails, but that test checks that DEBUG is set to False for a deployed project. This isn’t a live deployment, so I was expecting that test to fail.

BlogMaker Lite, on Django 5.0

Now let’s upgrade Django to 5.0, and see if the tests still pass:

(b_env)$ git checkout -b try_django_50
(b_env)$ pip install --upgrade django==5.0
Collecting django==5.0
  ...
Successfully installed asgiref-3.7.2 django-5.0
(b_env)$ python manage.py runserver
...
Django version 5.0, using settings 'blog.settings'
...

I’m making a new branch for this work, so it’s easy to move forward with the upgrade or roll back to the 4.1 version of the project.

Installation of Django 5.0 was successful. I also restarted the development server, so it’s using the new version of Django to serve requests.

Now we can run the tests again. The test script has a flag, —-flush-db, that flushes the database before running the tests. This is useful when testing repeated deployments. I’ll use that here, because the tests look for only the data that’s created during a single run of the test suite:

(b_env)$ python test_deployed_app_functionality.py \
    --url http://localhost:8000 --flush-db
Flushing db before running test...
  Flushed db.

Testing functionality of deployed app at http://localhost:8000/...

  Checking anonymous home page...
  Checking empty anonmyous all_blogs page...
  Checking empty anonmyous latest_posts page...
  ...

The tests still pass, which means the basic functionality of this project works on Django 5.0.

Updating requirements

To conclude the upgrade process, we can now re-freeze the requirements:

(b_env)$ pip freeze > requirements.txt
(b_env)$ git commit -am "Upgraded to Django 5.0."
(b_env)$ git push origin try_django_50

I’ll follow this up by making a pull request and merging this change. The full PR includes updates to all the project’s dependencies, as well as Django. I also added .venv/ to the .gitignore file, so people using the more modern convention won’t accidentally commit their virtual environment.

Anyone who builds a new environment for the project will be using Django 5.0 from this point forward. If you already had a working version of the project, you could build a new virtual environment using the updated requirements, and you’d be using 5.0 as well.

Try new versions of Django early!

Many existing Django projects will run without errors on the newest version of Django. There are numerous stories about people taking really old Django projects and jumping straight to the latest version, without bothering to upgrade through all the intermediate versions. These upgrades often require some changes to the codebase, but not as many as people expect to see.

Many of the errors that people run into when upgrading to a new version are fairly minor. A quick skim of the release notes usually clarifies the changes that are needed in order to make the project run on the latest version. If you face a more significant issue during the upgrade process, the sooner you know about it the better. You’ll have more time to prepare a manageable upgrade plan.

Don’t put off trying new versions of Django. It’s often easier than you think it will be, and if it’s difficult for some reason you’ll want to know about that sooner rather than later.

Bonus: Deploying BlogMaker Lite

As a quick bonus, I’d like to show how quickly you can deploy many Django projects using django-simple-deploy, the deployment project I’ve been working on.

For example if you have an account on Fly.io and have their CLI installed, you can deploy BlogMaker Lite in just three steps. The first is installing django-simple-deploy:

(b_env)$ pip install django-simple-deploy
Collecting django-simple-deploy
...
Successfully installed django-simple-deploy-0.6.1 toml-0.10.2

Now we need to add simple_deploy to INSTALLED_APPS, in settings.py:

INSTALLED_APPS = [
    # My apps.
    'blogs',
    'users',

    # Third party apps.
    'django_bootstrap5',
    'simple_deploy',

    # Default django apps.
    'django.contrib.admin',
    ...
]

And finally, we make a call to simple_deploy, specifying the platform we want to deploy to:

(b_env)$ python manage.py simple_deploy --platform fly_io --automate-all
Configuring project for deployment...
...

The —automate-all flag tells simple_deploy to inspect the project, configure it for deployment to the specified platform, create the resources needed for deployment, commit changes, and call the platform’s deploy command. It runs migrations, and opens the project in a new browser tab:

django-simple-deploy is currently in the pre-1.0 phase, so it won’t work in all situations. But most of the difficult issues have been sorted out, so bugs that come up now should be easier to address. You can choose from three currently supported platforms: Fly.io, Platform.sh, and Heroku. If you’re curious about this project, please give it a try. If you run into any difficulties, feel free to open an issue.

Note: As with all deployment-focused work, please only try django-simple-deploy if you’re comfortable working with your platform’s dashboard, and ensuring all deployed resources have been destroyed in a timely manner.

Conclusions

New versions of popular frameworks like Django are exciting, but they can also mean a pile of work dealing with upgrades. Fortunately, upgrading Django projects to the latest version has gotten much more straightforward over the years.

If you’ve put off trying the latest version because you think it will be a lot of work, try setting up a test environment and upgrading your project. It just might go quicker than you think, and you’ll make all subsequent upgrades easier as well.

If you’re not familiar with the proposal process, I’ll highlight some of the most important things to be aware of.

Proposal deadline

The deadline for submitting a proposal is 12/18, “Anywhere on Earth”. If you haven’t heard this phrase before, it’s a way of defining deadlines so that no one has to think about time zones. If it’s still 12/18 anywhere on Earth, you can still submit a talk proposal. Practically, that means you have until next Monday night to submit a proposal, no matter where you are in the world.

Proposal resources

There are a number of resources on the PyCon US site that are relevant to speakers. Here’s a rundown of the most important pages, and what kind of information you’ll find on each:

• PyCon US home page: An overview of the conference.

• Proposal guidelines: An overview of the kinds of proposals you can submit. Also includes a timeline for the proposal process.

• Proposing a talk: Suggested topics, topics to avoid, and other advice about making a proposal.

• Submitting a proposal: Directions for actually submitting a talk proposal.

Kinds of talks

There are several kinds of proposals you can submit:

• Talks: Most talks are 30 minutes, although there are some 45-minute slots for important topics.

• Charlas: There’s a Spanish track at PyCon US called Charlas. If you speak Spanish, even if it’s not your first language, consider submitting a Charlas proposal.

• Tutorials: You can propose a 3-hour interactive workshop for one of the days preceding the main conference. These are a tremendous amount of work, and are compensated.

• Posters: You can propose a poster that will be on display during part of the conference. There’s also a designated time for people to meet with you and discuss your work.

What to talk about

PyCon is a wonderful event in no small part due to the balance that the talk committee aims for, and usually achieves. Their goal is to have a full range of topics in the talks that are chosen. There are technical talks at the beginner, intermediate, and expert levels. There are context-focused talks, where people share what they’re using Python for in their paid work, academic work, and volunteer work. There are numerous presentations about longstanding and newer open source projects.

In short, if you’re doing something you find interesting with Python, please consider submitting a proposal.

One thing I’d love to see discussed next year: How are people using AI assistants in real-world projects? There have been plenty of discussions about how AI assistants respond to specific prompts. But what is it like to do your daily professional work with whatever AI assistant you use? What is it good at? What do you avoid using it for? What changes are you seeing in it over time? Is their consensus on your team about how to use (and not use) AI assistants?

People are also tired of hearing about AI, so if you have a specific topic to discuss that has nothing to do with AI, please submit that as well!

Drafting a proposal

Once you have a topic in mind, you can draft a proposal. The PyCon website has a portal for submitting a proposal. It’s helpful to know the sections you’ll be asked to fill in before starting your proposal. Some of these are public, and some are kept confidential between you and the conference organizers.

If you want to submit a proposal, you’ll need to fill in the following sections:

• Title: A one-line title for your proposal.

• Session Type: For example, a 30-minute talk.

• Track: This is basically the same thing as the Session Type.

• Description: A public description of your talk. It’s in Markdown, and you can use up to 300 words.

• Notes: Any additional information you might need to share with organizers. This is not public.

• Additional Speaker: You can co-present a talk if you want.

• Outline: Provide a rough outline of what you aim to present, and how long you’ll spend on each part of the talk.

• Category: There are about 30 categories to choose from. Some examples are Applications of Python, Data Science, DevOps, Security, etc.

• Audience: Choose from four main audience descriptions: Just starting out, Some experience, Advanced experience, Community presentation. This is meant to help attendees choose appropriate talks to go to.

• Link to Slides: You can provide a link to your slides ahead of time if you want to, and the audience can follow along during your talk. This link is not public until after your talk has been accepted.

• Have you previously given this talk? If so, the organizers would like a link to the relevant information.

• Recording release: This allows PyCon to record talks, and share them online after the conference is over.

• Resources: You can upload resources that you want attendees to have access to. I believe this is mostly used for tutorials.

This might look like a lot, but it’s what the review committee needs in order to figure out which talks are well-thought out proposals, and which are just quick ideas that might not actually lead to a good talk. It also helps them put together a well-balanced talk schedule. The most important parts are the description and the outline; the rest are important things to know, but quick to fill in.

Note: Keep in mind that the first round of reviews are blind; reviewers will not see your name during the initial review process. This policy was put in place to keep the focus on the contents of the proposal, and avoid overemphasizing well-known speakers. Make sure you don’t include any personally identifying information in your proposal’s description.

Don’t fear rejection!

PyCon gets a lot of proposals, and the review committee needs to put together a balanced schedule of talks. You can submit an amazing proposal about how dictionaries work in Python, and if five other people submit similar proposals, most of those will have to be rejected.

There are many really good proposals that end up being rejected because of overlap like this. Many well-known and popular speakers have been quite open about how many rejections they’ve received; if your talk is rejected, please don’t feel too discouraged. You can always resubmit your proposal for a later conference.

Other thoughts

There’s a few more things to keep in mind if you’re considering submitting a proposal:

Be clear and specific

The review committee is looking for reasons to accept your talk, or at least move it on to the second round of reviews. Give them all the information they need to make their decision, without writing up a full script for the talk.

You can submit multiple proposals

Because there’s a bit of luck around whether other people are submitting a talk on a similar subject, it’s perfectly acceptable to submit more than one proposal. However, it’s much better to submit one quality proposal than several lower-quality proposals.

You can save a draft

You can save your proposal as a draft. It’s actually good to fill in all the boxes, and come back a day or so later to look over your proposal before submitting it. Just remember to come back and follow through on submitting!

Don’t submit an ad

PyCon is a well-established conference, and no one wants to go see an ad for a for-profit project, or for the company somebody works for. If you want to share an interesting aspect of a commercial project, that can be appropriate. Just don’t submit a proposal that’s a thinly-veiled ad.

You can share a preview link with someone

It can be helpful to have someone else look over your proposal. While you can copy-paste parts of your proposal into an email, you can also share a link to a preview of your actual proposal.

Conclusions

If you’ve never submitted a proposal before, it can seem like a daunting process. But if you have a clear sense of what you’d like to present, it shouldn’t take too long to draft a proposal. It’s mostly a description of less than 300 words, and an outline showing you have a sense of how long you’ll spend on each major topic in your presentation.

I can’t promise a close review of proposals this close to the deadline, but if you want a quick check-in about an idea or a proposal, I’m happy to share my initial reactions. I’ve given one talk at DjangoCon, so I’ve been through this process. I’ve also had talks rejected, so I can commiserate if your proposal isn’t accepted. If you want some quick feedback, feel free to reply to this email with your thoughts.

If you submit a proposal, I wish you luck. :)

The two most difficult things in programming are naming things, cache invalidation, and off-by-one errors.

If you’ve never heard this joke before, I’m happy to be the first to share it with you. :)

Many people who are newer to programming think they’re the only ones who make “simple” mistakes like being off by one when using an index. But this joke has persisted for decades because people make “simple” mistakes like this throughout their careers in programming.

I recently made an off-by-one error in my own work, which led to a deeper understanding of how CLI apps are designed. I was never clear about why so many CLI apps use zero-indexing when presenting choices, until using a different approach led to a bug in my current project.

Which app do you want to use?

I’m currently working on a project to automate Django deployments. One challenge in trying to automate deployment centers around the possibility that the user may already have a number of pre-existing apps on the target platform. It’s really important that we don’t accidentally do anything with those apps.

During the deployment process, we need to identify which remote app to deploy against. To do this, we need to present the user with a list of choices:

Looking for Fly.io app to deploy against...

*** Found multiple undeployed apps on Fly.io. ***
1: bold-silence-958
2: green-cherry-4581

Which app would you like to use? 1

We tell the user that we’re looking for the right app to deploy to on the remote server, hosted at Fly.io. We show them all the apps that don’t already have an active deployment. We then let them choose which app to deploy to.

This dialog is user-facing, so I first implemented it as a list that starts at 1. We’re not writing code here, so I didn’t see any reason to start the list at 0.

Confirming the user’s choice

After the user makes their choice, we need to show the name of the app that was selected. We need to get a final confirmation that we should deploy to that app, because we can cause serious problems if we push to the wrong remote project.

Here’s a simplified version of the code I used to present this list of choices, and get the final confirmation that we’ve identified the correct app:

msg = "\n*** Found multiple undeployed apps on Fly.io. ***"
for index, name in enumerate(app_names, start=1):
    msg += f"\n{index}: {name}"

msg += "\nWhich app would you like to use? "
selection = input(msg)

selected_name = app_names[int(selection)]

msg = f"You have selected {selected_name}. Is that correct?"
confirmed = get_confirmation(msg)

if confirmed:
    # Continue with deployment...

Many people are familiar with the use of enumerate() to access the index value on each pass through a loop. We typically want this enumeration to start at the default value of 0, because sequence indexes start at 0. But you can tell enumerate() to use any starting value you want, using the start argument. Here we use start=1 to present a list of choices starting at 1, rather than 0.

Can you spot the issue with this approach, as it’s implemented here?

A buggy selection

When I ran this code, it didn’t work quite right:

Looking for Fly.io app to deploy against...

*** Found multiple undeployed apps on Fly.io. ***
1: bold-silence-958
2: green-cherry-4581

You can cancel this configuration work by entering q.
Which app would you like to use? 1

You have selected green-cherry-4581. Is that correct? (yes|no)

I selected the first choice, bold-silence-958. But the program picked out the name green-cherry-4581.

What’s really happening

The root of this issue is the fact that I’m using the value the user entered as the index when pulling the name from a list of app names. But the list of choices the user sees starts at one, and the indexes for my actual Python list starts at 0:

msg = "\n*** Found multiple undeployed apps on Fly.io. ***"
for index, name in enumerate(app_names, start=1):
    msg += f"\n  {index}: {name}"

msg += "\nWhich app would you like to use? "
selection = input(msg)

selected_name = app_names[int(selection)]

When the list of names is presented, that start=1 argument matches bold-silence-958 with the index 1. But in the list app_names, index 1 refers to the second name in the list, green-cherry-4581. Whichever choice the user makes, we’ll always be one name away from the name they chose. And if they choose the last name shown, we’ll get an IndexError.

Two solutions

There are two solutions to this issue:

• We can present the list as it’s currently displayed, and subtract 1 from the selected index before pulling the name from the list.

• Or, we can present the list using numbers that start at 0, matching the actual list indexes.

I don’t have a strong reason to start numbering this list at 1. If this were a list of the best-selling apps in an app store, you’d probably want to start at 1. But these numbers are arbitrary; we’re just using numbers to avoid requiring the user to type in a name. I’d much rather have simple code where the numbers in the CLI match the numbers we’re working with, than deal with differing internal and external numbering systems. The code will be simpler, and people reading this block of code won’t have to think through potential off-by-one issues.

The final implementation

The final implementation is just as easy for the end user to work with, and easier for developers to think about as well:

Looking for Fly.io app to deploy against...

*** Found multiple undeployed apps on Fly.io. ***
0: bold-silence-958
1: green-cherry-4581

You can cancel this configuration work by entering q.
Which app would you like to use? 0

You have selected bold-silence-958. Is that correct? (yes|no)
y

A third-party CLI

As an example of an existing CLI that works like this, consider the CLI for Platform.sh. When you deploy a project to Platform.sh, you can open the project in a browser directly from the command line. The platform url command gives you a list of all the URLs associated with the deployed project, and asks you to choose one:

$ platform url
Enter a number to open a URL
  [0] https://main-bvxea6i-cgxycamxvcfus.us-2.platformsh.site/
  [1] http://main-bvxea6i-cgxycamxvcfus.us-2.platformsh.site/
 > 0

This is presented as a zero-indexed list. I always thought CLIs were written this way because programmers just tend to think in terms of counting from 0. It never occurred to me that these selections were being used as the actual index for pulling a value from a sequence.

I haven’t seen the backend for their CLI, but I would guess that they’re using the input value directly to pull from their list of available URLs.

A physical zero-indexed tool

Most people have probably used zero-indexing before they started programming, without realizing it.

Think of a ruler for a moment. A ruler is a zero-indexed tool:

Objects can measure less than 1 centimeter, so a ruler starts at 0, not 1. It’s not just rulers that start at 0, either. Just about anything we measure starts at 0: length, angles, weights, and volumes, to name a few.

When I was a math teacher, I watched students make off-by-one style errors on a regular basis whenever we’d work with rulers or protractors. Zero-indexed systems are quite natural, and are more common than many people realize.

Conclusions

I like how addressing a small bug can sometimes lead to a deeper understanding of a concept or implementation pattern. I especially like it when the debugging work helps explain something I’ve seen often, but not paid much attention to before.

Off-by-one errors will always be with us, and they’ll keep coming up in surprising ways.

We try to do a gift exchange most years. In past years someone would take pieces of paper, write everyone’s names on them, and pull them from a hat to see who each person would be paired up with. This lets everyone be more thoughtful about what they give. This year someone asked if I could write a program to match people up, without having anyone matched with a member of their immediate family.

This is the kind of task that’s often not worth automating, because you can easily spend more time writing the program than you actually save. But I had time one morning last week, and there’s a fun joke waiting on the other side of this work, so I said I’d be happy to give it a try.

First pass

This is the kind of program that’s been written thousands of times, so you could probably look it up and find a program someone else wrote and just run it against your own family’s names. But it’s also the kind of specific task that’s fun to play with without looking at other people’s work. I wanted to come up with something that works, share the results with our family, and then consider what more efficient solutions might look like.

First, we need a list of all the names of people participating in the gift exchange:

names = [
    "Sherri",
    "Eric",
    "Terri",
    "Joel",
    "Joshua",
    "Catherine",
    "Jasmine",
    "Alexandra",
    "Jeremiah",
]

I used Faker to generate this list of names, but inserted my own name into the list for a reason you’ll see in a moment.1

Matching constraints

It might actually be worth writing a program to deal with a small matching problem like this, because matching people while meeting a set of constraints can be hard to do manually. The only constraint in our gift exchange is that partners shouldn’t be assigned to give gifts to each other.

Here are the partners in my (fake) family:

partners = [
    ("Sherri", "Eric"),
    ("Terri", "Joel"),
    ("Joshua", "Catherine"),
    ("Jasmine", "Alexandra"),
]

The constraint is implemented as a list of tuples. If any match we make includes one of these pairs, we need to reject that match.

Failed attempts

I always like to be honest about the background work I did before a writeup like this, because it’s easy to give the impression that I came up with a working solution on my first attempt. That’s rarely the case in my work, and the same is true for most of the programmers I know.

In my very first attempt I had it in my head that I needed to make pairs of people. I thought we needed to have an even number of participants for this to work. When I was looking at the first set of results I realized only half the people were giving gifts, and half were receiving gifts. That was a little facepalm moment, but it’s also something I love about programming. You can take an idea you have for a small problem, and try implementing it without having to think it through too deeply. The immediate feedback you get from trying something is invaluable, and almost always leads to a better understanding of the problem you’re trying to solve.

In my second attempt, I wrote a big nested loop. The outer loop kept running until a good set of matches was found, and the inner loop ran as long as there were still names that needed to be matched. That approach could work, but I find it hard to reason about nested while loops. I got a working solution by moving the body of the loops into functions, which greatly simplified my reasoning about the code.

Verifying matches

My main approach is to identify a list of givers and receivers. I should end up with nine sentences like this:

Eric will give to Alexandra.

This pairing would work, because it doesn’t violate the constraints defined in partners.

We need a function to check whether a match works or not:

def check_match(match):
    if match in partners:
        return False
    if tuple(reversed(match)) in partners:
        return False
    return True

Even though we’re going to output sentences, the actual matches are going to be tuples like this:

('Catherine', 'Joshua')

In check_match(), we want to make sure the current match is not in the list partners. If it is, we return False. If the match is not in partners, we also need to check whether the reversed version of the tuple is in partners. We only return True if both of these checks pass.2

When writing a function like this, it’s good to make a quick check that it works as expected:

print(check_match(('Sherri', 'Eric')))
print(check_match(('Eric', 'Sherri')))
print(check_match(('Sherri', 'Terri')))

The first two should fail because these matches are in the list partners. The third match should work.

False
False
True

The function seems to work, so we can move on. (If this were part of a longer-term project, I’d move this code to a test.)

Matching everyone

Now we need a function that takes the list of names, and tries to match everyone. I’m sure there are a number of ways of solving the problem of matching people for the purpose of giving gifts, and I’m sure some approaches are better than others. But I’m matching nine people for a family event, so I’m not too worried about finding the best solution, or a mathematically elegant solution. A lot of times in programming, you don’t need to find the best solution. You just need to find something that works, and you can then refine your solution as needed.

I’m going to try the following approach:

• Shuffle the list of names.

• Have person A give to person B, person B give to person C, and so forth.

• That should take care of everyone except the first person and the last person. The first person has only given a gift, and the last has only received a gift. So, the last person will give a gift to the first person.

• If this results in a match that doesn’t work, we’ll start over with a new shuffling of the original list.

Attempting a set of matches

Now we can write a function that attempts to make a full set of matches. The function will be given a copy of the original list of names, so it’s free to modify its copy of names:

def attempt_match(names):
    random.shuffle(names)
    matches = []

    # Keep track of first giver.
    giver = names.pop()
    first_giver = giver

    while names:
        # Receiver is next name in list.
        receiver = names.pop()
        match = (giver, receiver)

        # Bail if it's a bad match.
        if not check_match(match):
            return False

        # Store match.
        matches.append(match)

        # Current receiver becomes next giver.
        giver = receiver

    # The last receiver gives to the first giver.
    match = (receiver, first_giver)
    if not check_match(match):
        return False

    # If we're still here, all matches are good.
    return matches

The comments in here should help make sense of this code. We first shuffle the names, so everyone is in a random order. We then make an empty list to store matches in.

We need to keep track of the first person assigned to give a gift, because they’ll need to be matched with the last person in the list when we’re all done.3 So before starting the loop, we call pop() to pull a name from the list. This is the first person assigned to giver, but we also assign that name to first_giver.

With the first giver identified, we start looping through the remaining names. We pull the next name, and assign that name to receiver. This allows us to make the first match, of the form (giver, receiver). We check this match, and return False if it fails the call to check_match(). If it passes that check, we append the tuple to matches. Before moving on to the next name, we assign the current receiver to giver, making them the next person to give a gift.4

The loop ends when there are no names left. At that point, the first giver and the final receiver need to be paired up. We check that match, and if it passes we can return the set of matches.

Finding a working set of matches

Now we’re ready to use these functions, and hopefully get a working set of matches.

num_attempts = 0
while num_attempts < 10:
    num_attempts += 1

    matches = attempt_match(names[:])
    if matches:
        # Print summary, and exit.
        print(num_attempts)
        summarize_matches(matches)
        sys.exit()

# Failed to find a good match.
print("Couldn't make a good set of matches.")

I’m not sure how hard it is to find a working set of matches, so I’m setting a limit of 10 attempts for now. On each pass through the main loop, we call attempt_match(). That function will either return False, or a good set of matches. If we have a good set of matches, we print a summary of the matches and exit. I’m always curious how many attempts it takes, so I’m also printing that number.

If the loop ends without making a good match, we display a clear message to that effect.

Here’s the summarize_matches() function:

def summarize_matches(matches):
    for giver, receiver in matches:
        print(f"{giver} will give to {receiver}.")

This works:

5
Jasmine will give to Joshua.
Joshua will give to Alexandra.
Alexandra will give to Joel.
Joel will give to Jeremiah.
Jeremiah will give to Eric.
Eric will give to Catherine.
Catherine will give to Sherri.
Sherri will give to Terri.

In this run, it took 5 attempts to find a good set of matches. Running the program a few times, I saw it generate a good set of matches in 1 attempt, and I saw it take up to 7 attempts.

Sharing with family

As soon as this worked, I copied the output text and sent it to our family’s group text. I was quite intentional about sending the very first set of matches that met our family’s constraints. I’ll admit a little temptation to look over the matches, and see if I think they’re “good”. But that feels pretty bad, and even for a small thing like a gift exchange you really want to keep the trust your family has in you to be impartial.

A bit of lightness…

There’s a great joke to be had here, and I wonder if anyone reading has been thinking of it. I rewrote the function summarize_matches() slightly:

def summarize_matches(matches):
    for giver, receiver in matches:
        print(f"{giver} will give to Eric.")

This generates some very agreeable output:

Jeremiah will give to Eric.
Jasmine will give to Eric.
Sherri will give to Eric.
Catherine will give to Eric.
Terri will give to Eric.
Alexandra will give to Eric.
Joel will give to Eric.

I wrote a message to the family group text saying there was an issue with the initial output, and pasted this in as the corrected output. My immediate family groaned and sent facepalm emojis, but I quickly got more appropriately amused responses from my extended family.

Improvements

Before reading about other implementations, I found a significantly better approach just by writing this post. A lot of times I find that a bit of writing lets me see things more clearly. Sometimes that comes from writing a post, sometimes it’s writing to a colleague, and sometimes it comes from writing documentation.

As I was writing out the explanation for attempt_match(), I realized I could work directly with list indexes, rather than using intermediate variables such as giver and receiver. This version requires much less code:

def attempt_match(names):
    random.shuffle(names)

    # Match each person with the next, then match the last
    #   person with the first.
    matches = [(names[i], names[i+1]) for i in range(len(names)-1)]
    matches.append((names[-1], names[0]))

    # Check all matches.
    if all([check_match(match) for match in matches]):
        return matches
    return False

What we’re really doing in the earlier approach, after shuffling names, is matching each item in names with the item that follows. That is, we’re matching names[i] with names[i+1]. This doesn’t work for the last name in the list; we instead match that name with the first name in the list.

The code above uses a comprehension to do most of this. This approach lets us make all the matches in just two lines of code.5

With the whole set of matches made, we can use another comprehension to check all the matches in one line of code. Consider this comprehension:

[check_match(match) for match in matches]

This calls check_match() for every match in the list matches. It generates a list of boolean values, one for each match in the set of matches that were just generated. The all() function returns True if all values in the list are True. If they are, we return the list matches. If any of these are False, there’s a bad match and we return False to indicate a failed attempt at making an acceptable set of matches.

Conclusions

This was an interesting little exercise. I was going to do a bit more reading about mathematically elegant solutions to this problem, but the final version of attempt_match() is clean enough to satisfy my curiosity about this kind of problem for now.

If I were writing a version of this program to serve a high-traffic site, I’d take the time to read up on mathematically efficient models of this kind of problem. For example if I were implementing the matching algorithm for a game lobby, I’d want to use the most efficient approach.

Have you written any fun little programs to make your family life a little better? If so, please share your story! :)

Leave a comment

Resources

You can find the code files from this post in the mostly_python GitHub repository.

>>> from faker import Faker
>>> fake = Faker()
>>> names = [fake.first_name() for _ in range(9)]
>>> names
['Sherri', 'Peter', 'Terri', ..., 'Jeremiah']

>>> match = ('Sherri', 'Eric')
>>> rev_match = reversed(match)
>>> rev_match
<reversed object at 0x104655bd0>
>>> tuple(rev_match)
('Eric', 'Sherri')

def attempt_match(names):
    ...
    while names:
        ...
        # Current receiver becomes next giver.
        giver = receiver
    
    # The last receiver gives to the first giver.
    match = (receiver, first_giver)
    if not check_match(match):
        return False

    # If we're still here, all matches are good.
    return matches

match = (giver, receiver)

    matches = [
        (names[i], names[i + 1])
        for i in range(len(names) - 1)
    ]
    matches.append((names[-1], names[0]))

    matches = [
        (names[i], names[i + 1])
        for i in range(-1, len(names) - 1)
    ]

As some of you may know, Substack began implementing a referral program earlier this summer. I didn’t enable the program right away, because I wanted to see how it worked before doing so. I’m happy to help people share my work, but I don’t ever want to come across as spammy or overly self-promotional.

At this point there’s already a lot of meaningful content on Mostly Python, and you might want to share what’s here with other programmers you know. The referral program is pretty simple, so I enabled it today. From this point forward if you share the overall newsletter, or any individual post, you’ll get a few perks:

• If you refer 3 people who end up subscribing, you’ll get a month of paid access.

• If you refer 5 people, you’ll get 3 months of paid access.

• If you refer 25 people, you’ll get a year of paid access.

If you’re already a paying subscriber, your subscription will be extended by the appropriate amount.

You can click any Share or Refer a friend link you see, and your referral will count toward these complimentary subscriptions. If you’ve referred a few people and don’t see a message about a complimentary subscription, please reply to any email you receive from Mostly Python, and I’ll help you get it sorted.

Share Mostly Python

I really enjoy writing about my work with Python, and what I’ve learned about the language and the community. I don’t like gamification and turning growth into a competition. If you know someone who would benefit from this newsletter, please share a post or the newsletter itself. If you have questions about the overall Substack referral program, see their FAQ.

Thank you for helping get the word out about Mostly Python!

Thanks,

Eric