Settings that change, and settings that don't change

MP #29: Understanding subtle behaviors with "shared" objects.

Note: I’ll be starting a new series in June about OOP (object-oriented programming) in Python. If you have any questions about OOP, or topics you’d like to see covered, please feel free to reply to this email and I’ll make a note to include it in the series. Thanks!

---

A reader recently wrote to ask about a behavior they couldn’t explain. They noticed it in a game they were working on, but the root issue applies in many contexts outside of game development as well.

I was particularly impressed with this reader’s ability to create a minimum reproducible example, rather than sharing their entire project. Small examples are much easier to reason about, and people are much more likely to answer your questions if you can provide a short working example that demonstrates the issue you’re trying to understand.

Individual settings

The example involved a game with aliens, where each alien has a speed setting and a direction setting. The speed and direction of all aliens in the game should be the same. Here’s the main game file:

# game.py

class Game:

    def __init__(self):
        self.alien_speed = 5
        self.alien_direction = 1

    def show_settings(self):
        print("\nIn Game:")
        print(f"  alien speed: {self.alien_speed}")
        print(f"  alien direction: {self.alien_direction}")

if __name__ == '__main__':
    game = Game()
    game.show_settings()

We have a class called Game. In the __init__() method, attributes are defined for the aliens’ speed and direction. We also write a method called show_settings(), which prints out the values for these two settings.

When this file is run, it makes a Game object and calls show_settings():

In Game:
  alien speed: 5
  alien direction: 1

Nothing here is surprising, but it’s good to make sure everything is working as it should. Now let’s add an alien to the game:

from alien import Alien

class Game:

    def __init__(self):
        self.alien_speed = 5
        self.alien_direction = 1

        self.alien = Alien(self)

    def show_settings(self):
        ...

if __name__ == '__main__':
    game = Game()
    game.show_settings()
    game.alien.show_settings()

We import the Alien class, which we’re about to write. In the game’s __init__() method, we make an attribute which will refer to one Alien object. When we create an Alien object, we’re going to pass it a reference to the game object, so the alien can have access to the overall game attributes. This is a little complex if you haven’t seen a structure like this before, but it makes for a nice way to share information between parts of a larger project.

Note that the self shown in bold here refers to an instance of the Game class:

self.alien = Alien(self)

This code is inside the Game class; any reference to self inside this class is a reference to an instance of the Game class. (When you see self inside the Alien class, that will refer to an instance of the Alien class.)

Here’s alien.py:

class Alien:

    def __init__(self, game):
        self.speed = game.alien_speed
        self.direction = game.alien_direction

    def show_settings(self):
        print("\nIn Alien:")
        print(f"  alien speed: {self.speed}")
        print(f"  alien direction: {self.direction}")

The main line to understand here is the definition of the __init__() method. The first parameter, self, is the typical self parameter that we see in most method definitions. This self refers to an instance of the Alien class. The game parameter receives the game object that was passed from game.py when the Alien object was created.

In the Alien class we define two parameters, self.speed and self.direction. These are set to the same values that were set in the Game class. We also have a show_settings() method, which displays the value of the settings in this class.

Running game.py now, here’s the output:

In Game:
  alien speed: 5
  alien direction: 1

In Alien:
  alien speed: 5
  alien direction: 1

This still looks quite reasonable. All of the settings match, because none of them have been changed.

Changing the settings

Now let’s change the settings in Game, and see if the alien picks up the changes.

Here’s the updated version of game.py:

from alien import Alien

class Game:

    ...
    def increase_speed(self):
        self.alien_speed += 1

    def change_direction(self):
        self.alien_direction *= -1

if __name__ == '__main__':
    game = Game()
    game.show_settings()
    game.alien.show_settings()

    game.increase_speed()
    game.change_direction()

    game.show_settings()
    game.alien.show_settings()

We now have methods that increase the speed and change the direction of the alien. These methods are called, and then all of the settings are shown again.

If you have a moment, consider what you think the output will be before reading further. We’re directly changing the values in Game, so its settings should change. But what will happen to the settings in Alien? Will they change along with the settings in Game, or will they stay the same?

Here’s the output:

In Game:
  alien speed: 5
  alien direction: 1

In Alien:
  alien speed: 5
  alien direction: 1

In Game:
  alien speed: 6
  alien direction: -1

In Alien:
  alien speed: 5
  alien direction: 1

The settings in Game are updated, as expected. The settings in Alien are not affected. Before explaining this behavior, let’s look at a slightly different example.

Grouped settings

Now let’s group the settings into a dictionary and see what happens.

Here’s the updated Game class:

class Game:

    def __init__(self):
        self.settings = {
            'alien_speed': 5,
            'alien_direction': 1
        }

        self.alien = Alien(self)

    def show_settings(self):
        print("\nIn Game:")
        print(f"  alien speed: {self.settings['alien_speed']}")
        print(f"  alien direction: {self.settings['alien_direction']}")

    def increase_speed(self):
        self.settings['alien_speed'] += 1

    def change_direction(self):
        self.settings['alien_direction'] *= -1

The main change here is in the __init__() method. The two settings are key-value pairs in the settings dictionary, instead of individual attributes of the Game class. The rest of the changes consist of syntax updates to access the values from this dictionary.

Here are the changes to Alien:

class Alien:

    def __init__(self, game):
        self.settings = game.settings

    def show_settings(self):
        print("\nIn Alien:")
        print(f"  alien speed: {self.settings['alien_speed']}")
        print(f"  alien direction: {self.settings['alien_direction']}")

The __init__() method is simpler, because all the values are contained in one object. When we want to access them in show_settings(), we pull each one from the settings dictionary.

When we run game.py now, are we going to see the same behavior we saw before, where the settings in Alien don’t track the changes made in Game? Or are the values in Alien going to change along with the changes made in Game? Why do you think so?

Here’s the new output:

In Game:
  alien speed: 5
  alien direction: 1

In Alien:
  alien speed: 5
  alien direction: 1

In Game:
  alien speed: 6
  alien direction: -1

In Alien:
  alien speed: 6
  alien direction: -1

The alien’s settings track the values from the overall game this time. Why does this happen?

Let’s look at some much smaller examples, and then come back to the game example.

Small examples

For each of these examples we’ll ask if both values change, or just one.

Numerical values

What do you think will happen here?

x = 5
y = x
x += 1

print(f'{x = }')
print(f'{y = }')

We assign x a numerical value, and then set y equal to x. We then increment x, and print each value.¹ Does y have the original value of 5, or is it incremented to 6 as well?

Here’s the output:

x = 6
y = 5

String values

What about this one?

x = 'Hello'
y = x
x += ' everyone!'

print(f'{x = }')
print(f'{y = }')

We assign x the string 'Hello', and then set y equal to x. We then add on to the string that x refers to. Do both variables refer to the full sentence, or does y still point to a single word?

Here’s the output:

x = 'Hello everyone!'
y = 'Hello'

Lists

What if x points to a list?

x = [1, 2, 3]
y = x
x.append(4)

print(f'{x = }')
print(f'{y = }')

We assign a list containing three numbers to x, and point y to that same list. We then append an additional value to the list that x points to. Is the new number in both lists, or just in the one that x points to?

Here’s the output:

x = [1, 2, 3, 4]
y = [1, 2, 3, 4]

This time y does track the changes made to x.²

Mutable and immutable objects

The explanation for all of this centers around whether an object is mutable or immutable. An object is mutable if its value can be changed without creating a new object. An object is immutable if its value can’t be changed.

A list is a perfect example of a mutable object. You create a list, and then change it in all kinds of ways. For example you might add items, remove items, or modify items in the list. None of these actions cause a new list to be created.

People are less familiar with immutable objects. An integer is an immutable object. Consider this code:

>>> x = 5
>>> x += 1

It looks like we’re “adding 1 to 5”. What Python really does is calculate the value 6, and then point x to 6. The variable x ends up pointing at a different object. You can see this by printing the id of x at each point in this example:

>>> x = 5
>>> id(x)
4354432016
>>> x += 1
>>> id(x)
4354432048

The id() function returns a unique identifier associated with an object. Usually, this refers to the object’s address in memory. Here, the id associated with x has changed, showing that it’s pointing to a different object than it originally was. You’ll see this same kind of behavior with any immutable object.

If you try this with a list, you’ll see that the id stays the same:

>>> x = [1, 2, 3]
>>> id(x)
4349072320
>>> x.append(4)
>>> id(x)
4349072320

You’ll see the same kind of behavior with any mutable object.

Understanding the original example

Let’s apply what we’ve seen in these smaller examples to the original example. In the original version of game.py, each setting is a variable that refers to an integer:

    def __init__(self):
        self.alien_speed = 5
        self.alien_direction = 1

The important thing to notice here is that each of these settings refers to an integer, which is an immutable object.

Here’s the relevant part of the original version of alien.py:

    def __init__(self, game):
        self.speed = game.alien_speed
        self.direction = game.alien_direction

Because both game.alien_speed and game.alien_direction refer to immutable objects (integers), the attributes self.speed and self.direction won’t track any changes made to the corresponding variables in game.py.

Here’s the code that changes the alien’s speed in game.py:

    def increase_speed(self):
        self.alien_speed += 1

Because integers are immutable, this code doesn’t really “change the value” of self.alien_speed. Instead, it points self.alien_speed at a different immutable object, the value 6. The attribute self.speed in the Alien class is still pointing at the immutable object that represents the integer 5.

The settings dictionary

Why did the alien’s settings update when we used a dictionary? Here’s the relevant part of game.py from that example:

    def __init__(self):
        self.settings = {
            'alien_speed': 5,
            'alien_direction': 1
        }

There are integers in here, but the attribute self.settings doesn’t point to those integers. It points to a dictionary, which is a mutable object.

Here’s the relevant part of alien.py:

    def __init__(self, game):
        self.settings = game.settings

The Alien attribute self.settings now points to the same dictionary object that self.settings in Game does:

Both settings attributes point to the same actual dictionary object.

Any changes that are made to the settings dictionary in either place will affect the dictionary in the other class.³

Conclusions

There are lots of little nooks and crannies in Python, as there are in any sufficiently developed programming language. It takes time to become aware of many of these subtleties, and understand why they exist. Subtle behaviors almost always come from trying to keep a balance between having a language that’s easy to understand, and one that has reasonable performance characteristics as well.

In next week’s post, we’ll look at an even more subtle aspect of integers that relates to what was discussed here.

Resources

You can find the code files from this post in the mostly_python GitHub repository.

1. If you haven’t seen the syntax f'{x = }', it’s a nice feature of f-strings. The equal sign tells Python to print the name of the variable x, followed by an equal sign, followed by its value. This is helpful for debugging, and looking at small examples.

2. It’s possible to make an example that seems to contradict this:

x = [1, 2, 3]
y = x
x = [1, 2, 3, 4]

print(f'{x = }')
print(f'{y = }')

Here’s the output:

x = [1, 2, 3, 4]
y = [1, 2, 3]

In this example we didn’t actually modify the list that x refers to. Instead we created an entirely new list, and pointed x to that new list. The original list hasn’t been modified, and y still points to it.

3. Keep in mind that you can make an example that seems to contradict this as well. One mistake would be to have Alien pull values from the dictionary too early:

    def __init__(self, game):
        self.speed = game.settings['alien_speed']
        self.direction = game.settings['alien_direction']

This would point the attributes self.speed and self.direction to the immutable integer objects that are in the settings dictionary at the moment the Alien object is created. But there would be no ongoing connection with the settings dictionary. Any changes to the dictionary would not be carried back to these attributes.

When you use this kind of structure, it’s important to only pull values from the dictionary when you’re just about to use them.