Storing heartbeats from 250,000 screens with Redis write coalescing

Backend Live Coding November 13, 2026 python

250,000 screens / 60 s is about 4,000 writes per second for heartbeats alone.

python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# ======================================================================
# Q: Every screen sends a heartbeat every 60 seconds. How do you store it?
#
# 250,000 screens / 60 s is about 4,000 writes per second for heartbeats alone.
# One UPDATE per heartbeat in PostgreSQL creates dead tuples, autovacuum load
# and locks. I write the heartbeat to Redis and a periodic task moves the data
# to the database with a bulk update every few minutes (write coalescing).
# "Offline" then becomes a fast query in Redis. If a screen stays offline, I
# notify the customer (email/Slack).
# ======================================================================

# in the heartbeat endpoint: O(1), no database
redis.zadd('screens:last_seen', {screen_id: time.time()})

# Celery beat every 2 minutes
@shared_task
def flush_heartbeats():
    cutoff = time.time() - 120
    rows = redis.zrangebyscore('screens:last_seen', cutoff, '+inf', withscores=True)
    for chunk in batched(rows, 1000):
        Screen.objects.bulk_update(
            [Screen(pk=int(pk), last_seen=datetime.fromtimestamp(ts, UTC)) for pk, ts in chunk],
            ['last_seen'])

# Which screens are offline? ZRANGEBYSCORE with a score less than now - 5 minutes