Calling an external API from Celery with retries and backoff

Backend Live Coding November 13, 2026 python

I pass an id and not a model instance (the object in the queue may be stale).

python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# ======================================================================
# Q: How do you call an external API from Celery with retries?
#
# I pass an id and not a model instance (the object in the queue may be stale).
# The task is idempotent: if it runs twice, the result is the same. With
# acks_late, if the worker dies, the task runs again. That is why idempotency is
# mandatory.
# ======================================================================

@shared_task(bind=True,
             autoretry_for=(ExternalAPIError, requests.RequestException),
             retry_backoff=True,          # 1s, 2s, 4s, 8s ...
             retry_backoff_max=600,
             retry_jitter=True,
             max_retries=8,
             acks_late=True)
def refresh_weather_widget(self, widget_id):
    widget = Widget.objects.get(pk=widget_id)   # we pass an id, not an object
    try:
        data = WeatherClient(settings.WEATHER_KEY).forecast(widget.city)
    except RateLimited as exc:
        raise self.retry(exc=exc, countdown=exc.retry_after)
    Widget.objects.filter(pk=widget_id).update(payload=data, refreshed_at=now())