Cursor pagination for lists with millions of rows

Backend Live Coding November 13, 2026 python

Offset pagination (LIMIT/OFFSET) becomes slow on large pages, because the database reads and discards all the previous rows, and it shows duplicate or missing records when new ones are inserted.

python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# ======================================================================
# Q: Pagination on a list with millions of records (e.g. proof-of-play logs)?
#
# Offset pagination (LIMIT/OFFSET) becomes slow on large pages, because the
# database reads and discards all the previous rows, and it shows duplicate or
# missing records when new ones are inserted. I use CursorPagination (keyset)
# with a stable ordering on an indexed field, and avoid COUNT(*) on huge tables.
# ======================================================================

class PlayLogPagination(CursorPagination):
    page_size = 100
    ordering = '-id'          # unique and indexed

# Produces: WHERE id < :last_seen_id ORDER BY id DESC LIMIT 101