There has been some discussion about Python Global Interpreter Lock (GIL) and races following this interesting article. The limited role of the GIL in preventing such data races can be understood simply through use the dis python module.

Review of GIL

From PythonWiki: “In CPython, the global interpreter lock, or GIL, is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once” (my emphasis). Hence GIL operates on a bytecode by bytecode basis.

How does Python code relate to bytecode?

The python module dis allows inspection of bytecode (see docs ). If we use the key function from the above article as an example:

import dis

counter = 0

def increase():
    global counter
    for i in range(0, 100000):
        counter = counter + 1
			
dis.dis(increase)

This produces following result:

  3           0 LOAD_GLOBAL              0 (range)
              2 LOAD_CONST               1 (0)
              4 LOAD_CONST               2 (100000)
              6 CALL_FUNCTION            2
              8 GET_ITER
        >>   10 FOR_ITER                12 (to 24)
             12 STORE_FAST               0 (i)

  4          14 LOAD_GLOBAL              1 (counter)
             16 LOAD_CONST               3 (1)
             18 BINARY_ADD
             20 STORE_GLOBAL             1 (counter)
             22 JUMP_ABSOLUTE           10
        >>   24 LOAD_CONST               0 (None)
             26 RETURN_VALUE

Key lines are 14 to 20: the operations of loading the value of the counter (LOAD GLOBAL), addition (BINARY ADD) and storing the result (STORE GLOBAL) are all separate bytecodes (operations). The GIL will not enforce that all three executed together in a thread before giving way to another!

How to fix the race

In the case shown in the article linked above, the race can be fixed by adding an explicit lock to be associated with use of the counter global variable:

from threading import Thread, Lock
from time import sleep

counter = 0
lock = Lock()

def increase():
    global counter
    for i in range(0, 100000):
        with lock:
            counter = counter + 1

threads = []
for i in range(0, 400):
    threads.append(Thread(target=increase))
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print(f'Final counter: {counter}')