Finding a Python memory leak in a worker with tracemalloc

Backend Live Coding November 13, 2026 python

I compare two snapshots and see which line makes allocations that are not freed.

python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# ======================================================================
# Q: The memory of a worker keeps growing. How do you find the memory leak?
#
# I compare two snapshots and see which line makes allocations that are not
# freed. Usual culprits in Django: a module-level dict used as an unbounded
# cache, DEBUG=True which keeps all queries in connection.queries, iterating a
# huge QuerySet without .iterator() which fills the result cache. Mitigation in
# Celery: worker_max_tasks_per_child or worker_max_memory_per_child, but that is
# a band-aid, not a solution.
# ======================================================================

import tracemalloc

tracemalloc.start(25)                      # keeps 25 frames per allocation
snap1 = tracemalloc.take_snapshot()

for _ in range(1000):
    process_batch()                        # suspect code

snap2 = tracemalloc.take_snapshot()
for stat in snap2.compare_to(snap1, 'lineno')[:10]:
    print(stat)
# pricing/cache.py:31: size=48.2 MiB (+48.1 MiB), count=410233 (+410000)

Related Posts

Python shallow copy vs deep copy
Python shallow copy vs deep copy
Python closures and late binding in lambdas
Python closures and late binding in lambdas
Storing heartbeats from 250,000 screens with Redis write coalescing
Storing heartbeats from 250,000 screens with Redis write coalescing
Writing a retry decorator with exponential backoff in Python
Writing a retry decorator with exponential backoff in Python
SQL: what a CTE is and how to find duplicate records
SQL: what a CTE is and how to find duplicate records
Threads vs processes vs asyncio: when to use each
Threads vs processes vs asyncio: when to use each