Table of Contents
Introduction
Scheduling background jobs is a common requirement in Django applications. Celery, combined with Celery Beat, is a popular way to run periodic tasks like sending notifications, cleaning up data, or syncing with external services. In a multi-vendor SaaS platform, each vendor may need their own schedule for the same task. Hardcoding these schedules is not scalable. This guide explains how to use Django, Celery, and Celery Beat to manage both simple and dynamic schedules, focusing on database-driven flexibility.
Step-by-Step Implementation
1. Simple Static Scheduling with Celery Beat
Celery Beat lets you define periodic tasks in your Django project settings or Celery configuration. Here are several practical examples:
Example 1: Every Minute
app.conf.beat_schedule = {
'task-every-minute': {
'task': 'myapp.tasks.example_task',
'schedule': crontab(minute='*'),
},
}
Example 2: Every 2 Hours
app.conf.beat_schedule = {
'task-every-2-hours': {
'task': 'myapp.tasks.example_task',
'schedule': crontab(minute=0, hour='*/2'),
},
}
Example 3: Daily at 3:30 AM
app.conf.beat_schedule = {
'task-daily-3-30': {
'task': 'myapp.tasks.example_task',
'schedule': crontab(minute=30, hour=3),
},
}
Example 4: Every Monday at 9:00 AM
app.conf.beat_schedule = {
'task-monday-9am': {
'task': 'myapp.tasks.example_task',
'schedule': crontab(minute=0, hour=9, day_of_week=1),
},
}
Example 5: Last Day of the Month at 23:00
app.conf.beat_schedule = {
'task-last-day-month': {
'task': 'myapp.tasks.example_task',
'schedule': crontab(minute=0, hour=23, day_of_month='last'),
},
}
These examples show how flexible Celery Beat is for scheduling tasks in Django. You can use the crontab helper to express many different time-based schedules.
2. Limitations of Hardcoding Schedule
Hardcoding schedules in your Django or Celery configuration works for a few tasks, but it quickly becomes a problem as your application grows. Here are some real limitations:
- No per-user or per-vendor customization: If you want different users or vendors to have their own schedules, you would need to add a new entry for each one and redeploy your code every time a change is needed.
- Difficult to update: Changing a schedule means editing code, testing, and redeploying. This is slow and error-prone.
- No self-service: Non-developers cannot change schedules. Support or operations teams must request changes from developers.
- Risk of mistakes: With many hardcoded schedules, it’s easy to introduce conflicts or duplicate entries.
- No runtime flexibility: You cannot enable, disable, or adjust schedules without a code change and restart.
Example scenario:
If you have 50 vendors and each wants a daily report at a different time, you would need 50 separate entries in your configuration. If a vendor wants to change their time, you must update the code and redeploy. This is not practical for a growing SaaS platform.
3. Dynamic Scheduling with django-celery-beat
django-celery-beat stores schedules in the database. You can create, update, or remove schedules at runtime using the Django admin or programmatically.
Install and set up:
pip install django-celery-beat
Add to INSTALLED_APPS and migrate:
INSTALLED_APPS = [
# ...
'django_celery_beat',
]
python manage.py migrate django_celery_beat
Create a schedule for each vendor:
from django_celery_beat.models import PeriodicTask, CrontabSchedule
import json
def schedule_vendor_task(vendor_id, hour, minute, params):
schedule, _ = CrontabSchedule.objects.get_or_create(
minute=minute,
hour=hour,
day_of_week='*',
day_of_month='*',
month_of_year='*',
timezone='UTC',
)
PeriodicTask.objects.update_or_create(
name=f'vendor-task-{vendor_id}',
defaults={
'task': 'myapp.tasks.vendor_task',
'crontab': schedule,
'args': json.dumps([vendor_id]),
'kwargs': json.dumps(params),
'enabled': True,
}
)
Example Celery task:
from celery import shared_task
@shared_task
def vendor_task(vendor_id, **params):
# Task logic here
pass
Mermaid Diagram:
graph TD;
A[Vendor] -->|Set Schedule| B[Database]
B -->|Schedules| C[django-celery-beat]
C -->|Triggers| D[Celery Worker]
D -->|Runs Task| E[Customer Notification]
With this setup, each vendor can have their own schedule, and you can manage everything from the Django admin or with code.
Conclusion
Django, Celery, and Celery Beat make it easy to run scheduled jobs. For multi-vendor or dynamic needs, use django-celery-beat to store schedules in the database. This approach is flexible and maintainable and works well for SaaS platforms where each user or vendor may need their own schedule. Avoid hardcoding schedules, and let your application adapt as requirements change.
Recent Posts
