Skip Navigation
Show nav
Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
    • .NET
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
Hide categories

Categories

  • Heroku Architecture
    • Compute (Dynos)
      • Dyno Management
      • Dyno Concepts
      • Dyno Behavior
      • Dyno Reference
      • Dyno Troubleshooting
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Developer Tools
    • Command Line
    • Heroku VS Code Extension
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery & Integration (Heroku Flow)
    • Continuous Integration
  • Language Support
    • Node.js
      • Working with Node.js
      • Troubleshooting Node.js Apps
      • Node.js Behavior in Heroku
    • Ruby
      • Rails Support
      • Working with Bundler
      • Working with Ruby
      • Ruby Behavior in Heroku
      • Troubleshooting Ruby Apps
    • Python
      • Working with Python
      • Background Jobs in Python
      • Python Behavior in Heroku
      • Working with Django
    • Java
      • Java Behavior in Heroku
      • Working with Java
      • Working with Maven
      • Working with Spring Boot
      • Troubleshooting Java Apps
    • PHP
      • PHP Behavior in Heroku
      • Working with PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
    • .NET
      • Working with .NET
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
      • Migrating to Heroku Postgres
    • Heroku Key-Value Store
    • Apache Kafka on Heroku
    • Other Data Stores
  • AI
    • Model Context Protocol
    • Vector Database
    • Heroku Inference
      • Inference API
      • Quick Start Guides
      • AI Models
      • Inference Essentials
    • Working with AI
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
      • Single Sign-on (SSO)
    • Private Spaces
      • Infrastructure Networking
    • Compliance
  • Heroku Enterprise
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • Language Support
  • Python
  • Background Jobs in Python
  • Background Tasks in Python with RQ

Background Tasks in Python with RQ

English — 日本語に切り替える

Last updated October 11, 2024

Table of Contents

  • Configuration
  • Create a worker
  • Queuing jobs
  • Deployment
  • Troubleshooting

RQ (Redis Queue) makes it easy to add background tasks to your Python applications on Heroku. RQ uses a Redis database as a queue to process background jobs. To get started using RQ, you need to configure your application and then run a worker process in your application.

Configuration

To setup RQ and its dependencies, install it using pip:

$ pip install rq

Be sure to add rq to your requirements.txt file as well.

Create a worker

Now that we have everything we need to create a worker process, let’s create one.

Create a file called worker.py. This module will listen to queued tasks and process them as they are received.

import os

import redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDIS_URL', 'redis://localhost:6379')

conn = redis.from_url(redis_url)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

Now you can run your new worker process:

$ python worker.py

Queuing jobs

To send a job to your new Redis Queue, you need to send your jobs to Redis in your code. Let’s pretend we have a blocking function in an external module, utils.py:

import requests

def count_words_at_url(url):
    resp = requests.get(url)
    return len(resp.text.split())

In your application, create a RQ queue:

from rq import Queue
from worker import conn

q = Queue(connection=conn)

And enqueue the function call:

from utils import count_words_at_url

result = q.enqueue(count_words_at_url, 'http://heroku.com')

The blocking function will automatically be executed in the background worker process.

Deployment

To deploy your new worker system to Heroku, you need to add the worker process to the Procfile in the root of the project.

worker: python worker.py

Additionally, provision an instance of the Heroku Key-Value Store (heroku-redis) add-on and deploy with a git push.

$ heroku addons:create heroku-redis
----> Adding heroku-redis to example-app... done, v10 (free)

$ git push heroku main
Counting objects: 5, done.
Delta compression using up to 4 threads.
...

Once everything’s pushed up, scale your workers according to your needs:

$ heroku scale worker=1
Scaling worker processes... done, now running 1

Troubleshooting

View worker process output by filtering the logs with the -p flag and the name of the worker process type.

$ heroku logs -t -p worker

The worker process can be manually invoked for further isolation.

$ heroku run worker
Running worker attached to terminal... up, run.1

Keep reading

  • Background Jobs in Python

Feedback

Log in to submit feedback.

Using Celery on Heroku Scheduled Jobs with Custom Clock Processes in Python with APScheduler

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure
  • .NET

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing
  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Github
  • LinkedIn
  • © 2025 Salesforce, Inc. All rights reserved. Various trademarks held by their respective owners. Salesforce Tower, 415 Mission Street, 3rd Floor, San Francisco, CA 94105, United States
  • heroku.com
  • Legal
  • Terms of Service
  • Privacy Information
  • Responsible Disclosure
  • Trust
  • Contact
  • Cookie Preferences
  • Your Privacy Choices