rq/docs/patterns/index.md

98 lines
2.3 KiB
Markdown
Raw Normal View History

2018-03-03 00:21:44 +00:00
---
title: "RQ: Using RQ on Heroku"
layout: patterns
---
## Using RQ on Heroku
To setup RQ on [Heroku][1], first add it to your
`requirements.txt` file:
redis>=3
rq>=0.13
2018-03-03 00:21:44 +00:00
Create a file called `run-worker.py` with the following content (assuming you
are using [Heroku Data For Redis][2] with Heroku):
2018-03-03 00:21:44 +00:00
2021-12-07 12:35:56 +00:00
```python
2018-03-03 00:21:44 +00:00
import os
import redis
2018-03-03 00:21:44 +00:00
from redis import Redis
from rq import Queue, Connection
from rq.worker import HerokuWorker as Worker
2018-03-03 00:21:44 +00:00
listen = ['high', 'default', 'low']
redis_url = os.getenv('REDIS_URL')
2018-03-03 00:21:44 +00:00
if not redis_url:
raise RuntimeError("Set up Heroku Data For Redis first, \
make sure its config var is named 'REDIS_URL'.")
conn = redis.from_url(redis_url)
2018-03-03 00:21:44 +00:00
if __name__ == '__main__':
with Connection(conn):
worker = Worker(map(Queue, listen))
worker.work()
2021-12-07 12:35:56 +00:00
```
2018-03-03 00:21:44 +00:00
Then, add the command to your `Procfile`:
2018-03-03 00:21:44 +00:00
worker: python -u run-worker.py
Now, all you have to do is spin up a worker:
2021-12-07 12:35:56 +00:00
```console
2018-03-03 00:21:44 +00:00
$ heroku scale worker=1
2021-12-07 12:35:56 +00:00
```
2018-03-03 00:21:44 +00:00
If the from_url function fails to parse your credentials, you might need to do so manually:
2021-12-07 12:35:56 +00:00
```console
conn = redis.Redis(
host=host,
password=password,
port=port,
ssl=True,
ssl_cert_reqs=None
)
2021-12-07 12:35:56 +00:00
```
The details are from the 'settings' page of your Redis add-on on the Heroku dashboard.
and for using the cli:
2021-12-07 12:35:56 +00:00
```console
rq info --config rq_conf
2021-12-07 12:35:56 +00:00
``````
Where the rq_conf.py file looks like:
2021-12-07 12:35:56 +00:00
```console
REDIS_HOST = "host"
REDIS_PORT = port
REDIS_PASSWORD = "password"
REDIS_SSL = True
REDIS_SSL_CA_CERTS = None
REDIS_DB = 0
REDIS_SSL_CERT_REQS = None
2021-12-07 12:35:56 +00:00
``````
2018-03-03 00:21:44 +00:00
## Putting RQ under foreman
[Foreman][3] is probably the process manager you use when you host your app on
Heroku, or just because it's a pretty friendly tool to use in development.
When using RQ under `foreman`, you may experience that the workers are a bit quiet sometimes. This is because of Python buffering the output, so `foreman`
cannot (yet) echo it. Here's a related [Wiki page][4].
2018-03-03 00:21:44 +00:00
Just change the way you run your worker process, by adding the `-u` option (to
force stdin, stdout and stderr to be totally unbuffered):
worker: python -u run-worker.py
[1]: https://heroku.com
[2]: https://devcenter.heroku.com/articles/heroku-redis
2018-03-03 00:21:44 +00:00
[3]: https://github.com/ddollar/foreman
[4]: https://github.com/ddollar/foreman/wiki/Missing-Output
[5]: https://elements.heroku.com/addons/heroku-redis