Kodeclik Blog

How to Loop Through All Variables in Python Memory Space

Suppose you have a Python program containing numerous variables and you wish to loop through all variables in memory so far and print their values out. How can you do that?

Consider:

# Create variables
x = 10
y = "hello"
z = [1, 2, 3]

How can you loop through and print these variables? One idea is to create a list of these variables and use eval to print their values:

# Create variables
x = 10
y = "hello"
z = [1, 2, 3]

# Store variable names in a list
variable_names = ['x', 'y', 'z']

# Loop through the list of variable names and print their values
for name in variable_names:
    print(f"{name}: {eval(name)}")

This approach uses the eval() function to get the value of each variable by its name string. The code maintains the original variable declarations but provides a more controlled way to iterate through specific variables you want to inspect.

Note that using eval() should be done cautiously in production code, as it can execute arbitrary Python expressions. However, in this controlled example where we know exactly what variable names we're evaluating, it's safe to use.

The output will be:

x: 10
y: hello
z: [1, 2, 3]

as desired.

But this is a bit artificial. You have to take the trouble of cataloging all variables and place them in a list for this approach to work. What if you could ask Python to do the book-keeping for you?

Python provides several built-in functions to inspect variables in memory. The most common approaches use globals(), locals(), and dir() functions. Let's explore these with practical examples.

Use globals() to loop through all variables in memory

Here is an approach that does not refer to the specific variables used. It uses the globals() function in Python which returns a dictionary containing all variables and symbols in the current program's global scope. This dictionary, known as the global symbol table, maintains essential information about the program and provides access to all globally defined variables, functions, and other objects.

How to loop through all variables in memory

# Create some variables
x = 10
y = "hello"
z = [1, 2, 3]

# Loop through a copy of global variables
for variable_name, value in dict(globals()).items():
    if not variable_name.startswith('__'):
        print(f"{variable_name}: {value}")

The interesting part in the above program happens in the loop, where dict(globals()) creates a dictionary of the entire global namespace. The loop then examines each name-value pair in this dictionary using the .items() method, which returns both the variable name and its corresponding value.

Note that we filter out system variables (those starting with double underscores) using a simple condition, showing only user-defined variables. When run, it will display each variable name alongside its value, effectively giving us a peek into what's currently stored in our program's memory. This technique can be particularly useful during debugging or when you need to inspect the state of your program's variables at a specific point in execution.

The output is:

x: 10
y: hello
z: [1, 2, 3]

Note that the code above WILL NOT work if you had not converted globals() to a dictionary using the dict() constructor. Thus, if your code was:

# Create some variables
x = 10
y = "hello"
z = [1, 2, 3]

# Loop through all global variables
for variable_name, value in globals().items():
    if not variable_name.startswith('__'):  # Skip system variables
        print(f"{variable_name}: {value}")

You will get an error such as:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    for variable_name, value in globals().items():
RuntimeError: dictionary changed size during iteration

The error occurs because the loop itself creates iterator variables (variable_name and value) which get added to the global namespace while you're iterating through it!

So in our earlier code the dict() constructor creates a fresh copy of globals() just before the loop iterator variables are created and uses it for the iteration.

Another approach to avoid this problem is to pre-declare the loop variables, like so:

# Create some variables
x = 10
y = "hello"
z = [1, 2, 3]

# Pre-declare the loop variables
variable_name = None
value = None

# Now loop through global variables
for variable_name, value in globals().items():
    if not variable_name.startswith('__'):
        print(f"{variable_name}: {value}")

Here note that variable_name and value are predeclared so that inside the loop it will not seem as if we are adding new variables. If we run this program, we get:

x: 10
y: hello
z: [1, 2, 3]
variable_name: z
value: z

Note that the loop variables are now part of the global list of variables and so they are printed as well. After the loop finishes its last iteration (which was for variable 'z'), these variables retain their final values. The last iteration processed 'z', so variable_name contains 'z' and value also contains 'z'.

Printing system variables

What happens if we remove the not variable_name.startswith('__') check? Lets try it and see. Our program is now:

# Create some variables
x = 10
y = "hello"
z = [1, 2, 3]

# Pre-declare the loop variables
variable_name = None
value = None

# Now loop through all variables
for variable_name, value in globals().items():
        print(f"{variable_name}: {value}")

The output will be (yours will vary):

__name__: __main__
__doc__: None
__package__: None
__loader__: <_frozen_importlib_external.SourceFileLoader object at 0x7f37c246adc0>
__spec__: None
__annotations__: {}
__builtins__: <module 'builtins' (built-in)>
__file__: main.py
__cached__: None
x: 10
y: hello
z: [1, 2, 3]
variable_name: z
value: z

Note that there are a huge number of variables being printed, none of which were created by you but which are used internally by your Python interpreter. This is why it is good to “hide” these variables using our check.

Creating Variables Dynamically

One of the most practical aspects of globals() is that it returns a writable dictionary, meaning you can not only inspect but also modify global variables through it! For example, if you have a global variable 'age = 23', you can modify it using globals()['age'] = 25, which will update the value in the global scope. This functionality is particularly useful when you need to dynamically access or modify global variables, though it should be used judiciously as relying heavily on global variables is generally not considered best practice in Python programming.

For instance, let us create 20 variables automatically:

# Create 20 variables automatically
for i in range(20):
    globals()[f'variable_{i}'] = i * 10

# Print all created variables
for name, value in dict(globals()).items():
    if name.startswith('variable_'):
        print(f"{name} = {value}")

The output will be:

variable_0 = 0
variable_1 = 10
variable_2 = 20
variable_3 = 30
variable_4 = 40
variable_5 = 50
variable_6 = 60
variable_7 = 70
variable_8 = 80
variable_9 = 90
variable_10 = 100
variable_11 = 110
variable_12 = 120
variable_13 = 130
variable_14 = 140
variable_15 = 150
variable_16 = 160
variable_17 = 170
variable_18 = 180
variable_19 = 190

Imagine how much more cumbersome it would have been to do this in your Python program manually.

You would have had to take the output and make that to be the program!

Inspecting locals() and looping through them

Thus far we have focused on global variable inspection. Let us look at local variables now! Consider the program:

def nested_example():
    x = 100
    y = "test"
    
    def inner_function():
        inner_var = 42
        print("\nLocals in inner function:")
        for name, val in locals().items():
            print(f"{name}: {val}")
    
    print("\nLocals in outer function:")
    for name, val in locals().items():
        print(f"{name}: {val}")
    
    inner_function()

# Call the functions
example_function()
nested_example()

When you run this program, it shows how locals() captures variables in different scopes. The first function demonstrates basic local variable inspection, while the nested function shows how locals() works in different levels of scope. locals() is particularly useful for debugging or when you need to inspect variables within a specific function's scope.

Note that unlike globals(), locals() returns a dictionary that should be treated as read-only in CPython. While globals() can be modified to affect global variables, modifying the dictionary returned by locals() may not affect the actual local variables.

The output will be:

Local variables inside the function:
name: Alice
age: 25
scores: [90, 85, 88]

Locals in outer function:
x: 100
y: test
inner_function: <function nested_example.<locals>.inner_function at 0x7f345dfe0e50>

Locals in inner function:
inner_var: 42

In the first function example_function(), locals() displays all variables defined within that function's scope. We see three local variables: name, age, and scores, along with their respective values. These variables exist only within this function's scope.

The nested example is more interesting as it shows how locals() behaves in nested scopes. In the outer function nested_example(), locals() shows three items: the variables x and y, plus inner_function which appears as a function object (with its memory address). This demonstrates that Python treats the inner function itself as a local variable in the outer function's scope.

Finally, when locals() is called inside inner_function(), it only shows inner_var: 42, as that's the only variable defined in that innermost scope. This illustrates Python's scoping rules, where each function maintains its own local namespace.

When will you need to loop through all variables in memory?

While it's generally discouraged to rely on global variable inspection, there are legitimate use cases. One common scenario is in debugging and testing environments where you need to inspect the program's state. Another use case is in scientific computing or data analysis where you might need to track multiple variables during an experiment.

In general, remember that while these tools exist, they should be used sparingly. If you find yourself frequently needing to inspect or modify global variables, consider refactoring your code to use classes or other data structures that better encapsulate your data and provide clearer interfaces for manipulation.

Enjoy this blogpost? Want to learn Python with us? Sign up for 1:1 or small group classes.