rq/CHANGES.md

737 lines
31 KiB
Markdown
Raw Normal View History

### RQ unreleased
* Dropped support for Python 3.6
2023-06-19 23:28:45 +00:00
### RQ 1.15.1 (2023-06-20)
* Fixed a bug that may cause a crash when cleaning intermediate queue. Thanks @selwin!
* Fixed a bug that may cause canceled jobs to still run dependent jobs. Thanks @fredsod!
2023-05-23 23:35:03 +00:00
### RQ 1.15 (2023-05-24)
* Added `Callback(on_stopped='my_callback)`. Thanks @eswolinsky3241!
2023-05-27 02:26:09 +00:00
* `Callback` now accepts dotted path to function as input. Thanks @rishabh-ranjan!
2023-05-23 23:35:03 +00:00
* `queue.enqueue_many()` now supports job dependencies. Thanks @eswolinsky3241!
* `rq worker` CLI script now configures logging based on `DICT_CONFIG` key present in config file. Thanks @juur!
* Whenever possible, `Worker` now uses `lmove()` to implement [reliable queue pattern](https://redis.io/commands/lmove/). Thanks @selwin!
* Require `redis>=4.0.0`
2023-05-23 23:35:03 +00:00
* `Scheduler` should only release locks that it successfully acquires. Thanks @xzander!
* Fixes crashes that may happen by changes to `as_text()` function in v1.14. Thanks @tchapi!
2023-05-27 02:26:09 +00:00
* Various linting, CI and code quality improvements. Thanks @robhudson!
2023-05-23 23:35:03 +00:00
2023-05-05 14:00:49 +00:00
### RQ 1.14.1 (2023-05-05)
* Fixes a crash that happens if Redis connection uses SSL. Thanks @tchapi!
* Fixes a crash if `job.meta()` is loaded using the wrong serializer. Thanks @gabriels1234!
2023-05-01 05:43:45 +00:00
### RQ 1.14.0 (2023-05-01)
2023-05-01 10:54:20 +00:00
* Added `WorkerPool` (beta) that manages multiple workers in a single CLI. Thanks @selwin!
2023-05-01 05:43:45 +00:00
* Added a new `Callback` class that allows more flexibility in declaring job callbacks. Thanks @ronlut!
* Fixed a regression where jobs with unserializable return value crashes RQ. Thanks @tchapi!
* Added `--dequeue-strategy` option to RQ's CLI. Thanks @ccrvlh!
* Added `--max-idle-time` option to RQ's worker CLI. Thanks @ronlut!
* Added `--maintenance-interval` option to RQ's worker CLI. Thanks @ronlut!
* Fixed RQ usage in Windows as well as various other refactorings. Thanks @ccrvlh!
* Show more info on `rq info` CLI command. Thanks @iggeehu!
* `queue.enqueue_jobs()` now properly account for job dependencies. Thanks @sim6!
* `TimerDeathPenalty` now properly handles negative/infinite timeout. Thanks @marqueurs404!
2023-02-19 09:30:07 +00:00
### RQ 1.13.0 (2023-02-19)
* Added `work_horse_killed_handler` argument to `Worker`. Thanks @ronlut!
* Fixed an issue where results aren't properly persisted on synchronous jobs. Thanks @selwin!
* Fixed a bug where job results are not properly persisted when `result_ttl` is `-1`. Thanks @sim6!
* Various documentation and logging fixes. Thanks @lowercase00!
* Improve Redis connection reliability. Thanks @lowercase00!
* Scheduler reliability improvements. Thanks @OlegZv and @lowercase00!
* Fixed a bug where `dequeue_timeout` ignores `worker_ttl`. Thanks @ronlut!
* Use `job.return_value()` instead of `job.result` when processing callbacks. Thanks @selwin!
* Various internal refactorings to make `Worker` code more easily extendable. Thanks @lowercase00!
* RQ's source code is now black formatted. Thanks @aparcar!
2023-01-15 11:21:40 +00:00
### RQ 1.12.0 (2023-01-15)
* RQ now stores multiple job execution results. This feature is only available on Redis >= 5.0 Redis Streams. Please refer to [the docs](https://python-rq.org/docs/results/) for more info. Thanks @selwin!
* Improve performance when enqueueing many jobs at once. Thanks @rggjan!
* Redis server version is now cached in connection object. Thanks @odarbelaeze!
* Properly handle `at_front` argument when jobs are scheduled. Thanks @gabriels1234!
* Add type hints to RQ's code base. Thanks @lowercase00!
* Fixed a bug where exceptions are logged twice. Thanks @selwin!
* Don't delete `job.worker_name` after job is finished. Thanks @eswolinsky3241!
2022-09-25 01:44:51 +00:00
### RQ 1.11.1 (2022-09-25)
* `queue.enqueue_many()` now supports `on_success` and on `on_failure` arguments. Thanks @y4n9squared!
* You can now pass `enqueue_at_front` to `Dependency()` objects to put dependent jobs at the front when they are enqueued. Thanks @jtfidje!
* Fixed a bug where workers may wrongly acquire scheduler locks. Thanks @milesjwinter!
* Jobs should not be enqueued if any one of it's dependencies is canceled. Thanks @selwin!
* Fixed a bug when handling jobs that have been stopped. Thanks @ronlut!
* Fixed a bug in handling Redis connections that don't allow `SETNAME` command. Thanks @yilmaz-burak!
2022-07-31 08:56:59 +00:00
### RQ 1.11 (2022-07-31)
2022-07-31 10:00:37 +00:00
* This will be the last RQ version that supports Python 3.5.
2022-07-31 08:56:59 +00:00
* Allow jobs to be enqueued even when their dependencies fail via `Dependency(allow_failure=True)`. Thanks @mattchan-tencent, @caffeinatedMike and @selwin!
* When stopped jobs are deleted, they should also be removed from FailedJobRegistry. Thanks @selwin!
* `job.requeue()` now supports `at_front()` argument. Thanks @buroa!
* Added ssl support for sentinel connections. Thanks @nevious!
* `SimpleWorker` now works better on Windows. Thanks @caffeinatedMike!
* Added `on_failure` and `on_success` arguments to @job decorator. Thanks @nepta1998!
* Fixed a bug in dependency handling. Thanks @th3hamm0r!
* Minor fixes and optimizations by @xavfernandez, @olaure, @kusaku.
2021-12-07 12:45:48 +00:00
### RQ 1.10.1 (2021-12-07)
* **BACKWARDS INCOMPATIBLE**: synchronous execution of jobs now correctly mimics async job execution. Exception is no longer raised when a job fails, job status will now be correctly set to `FAILED` and failure callbacks are now properly called when job is run synchronously. Thanks @ericman93!
2021-12-07 12:45:48 +00:00
* Fixes a bug that could cause job keys to be left over when `result_ttl=0`. Thanks @selwin!
* Allow `ssl_cert_reqs` argument to be passed to Redis. Thanks @mgcdanny!
* Better compatibility with Python 3.10. Thanks @rpkak!
* `job.cancel()` should also remove itself from registries. Thanks @joshcoden!
* Pubsub threads are now launched in `daemon` mode. Thanks @mik3y!
2021-09-09 11:31:21 +00:00
### RQ 1.10.0 (2021-09-09)
* You can now enqueue jobs from CLI. Docs [here](https://python-rq.org/docs/#cli-enqueueing). Thanks @rpkak!
* Added a new `CanceledJobRegistry` to keep track of canceled jobs. Thanks @selwin!
* Added custom serializer support to various places in RQ. Thanks @joshcoden!
* `cancel_job(job_id, enqueue_dependents=True)` allows you to cancel a job while enqueueing its dependents. Thanks @joshcoden!
* Added `job.get_meta()` to fetch fresh meta value directly from Redis. Thanks @aparcar!
* Fixes a race condition that could cause jobs to be incorrectly added to FailedJobRegistry. Thanks @selwin!
* Requeueing a job now clears `job.exc_info`. Thanks @selwin!
* Repo infrastructure improvements by @rpkak.
* Other minor fixes by @cesarferradas and @bbayles.
2021-06-30 07:08:23 +00:00
### RQ 1.9.0 (2021-06-30)
* Added success and failure callbacks. You can now do `queue.enqueue(foo, on_success=do_this, on_failure=do_that)`. Thanks @selwin!
* Added `queue.enqueue_many()` to enqueue many jobs in one go. Thanks @joshcoden!
* Various improvements to CLI commands. Thanks @rpkak!
* Minor logging improvements. Thanks @clavigne and @natbusa!
2021-05-17 06:58:08 +00:00
### RQ 1.8.1 (2021-05-17)
* Jobs that fail due to hard shutdowns are now retried. Thanks @selwin!
* `Scheduler` now works with custom serializers. Thanks @alella!
* Added support for click 8.0. Thanks @rpkak!
* Enqueueing static methods are now supported. Thanks @pwws!
* Job exceptions no longer get printed twice. Thanks @petrem!
2021-03-31 01:45:17 +00:00
### RQ 1.8.0 (2021-03-31)
* You can now declare multiple job dependencies. Thanks @skieffer and @thomasmatecki for laying the groundwork for multi dependency support in RQ.
* Added `RoundRobinWorker` and `RandomWorker` classes to control how jobs are dequeued from multiple queues. Thanks @bielcardona!
* Added `--serializer` option to `rq worker` CLI. Thanks @f0cker!
* Added support for running asyncio tasks. Thanks @MyrikLD!
* Added a new `STOPPED` job status so that you can differentiate between failed and manually stopped jobs. Thanks @dralley!
* Fixed a serialization bug when used with job dependency feature. Thanks @jtfidje!
* `clean_worker_registry()` now works in batches of 1,000 jobs to prevent modifying too many keys at once. Thanks @AxeOfMen and @TheSneak!
* Workers will now wait and try to reconnect in case of Redis connection errors. Thanks @Asrst!
2020-11-29 09:41:56 +00:00
### RQ 1.7.0 (2020-11-29)
* Added `job.worker_name` attribute that tells you which worker is executing a job. Thanks @selwin!
* Added `send_stop_job_command()` that tells a worker to stop executing a job. Thanks @selwin!
* Added `JSONSerializer` as an alternative to the default `pickle` based serializer. Thanks @JackBoreczky!
* Fixes `RQScheduler` running on Redis with `ssl=True`. Thanks @BobReid!
2020-11-08 04:37:03 +00:00
### RQ 1.6.1 (2020-11-08)
* Worker now properly releases scheduler lock when run in burst mode. Thanks @selwin!
2020-11-08 04:17:20 +00:00
### RQ 1.6.0 (2020-11-08)
* Workers now listen to external commands via pubsub. The first two features taking advantage of this infrastructure are `send_shutdown_command()` and `send_kill_horse_command()`. Thanks @selwin!
* Added `job.last_heartbeat` property that's periodically updated when job is running. Thanks @theambient!
* Now horses are killed by their parent group. This helps in cleanly killing all related processes if job uses multiprocessing. Thanks @theambient!
* Fixed scheduler usage with Redis connections that uses custom parser classes. Thanks @selwin!
* Scheduler now enqueue jobs in batches to prevent lock timeouts. Thanks @nikkonrom!
* Scheduler now follows RQ worker's logging configuration. Thanks @christopher-dG!
2020-09-10 01:19:53 +00:00
### RQ 1.5.2 (2020-09-10)
* Scheduler now uses the class of connection that's used. Thanks @pacahon!
* Fixes a bug that puts retried jobs in `FailedJobRegistry`. Thanks @selwin!
* Fixed a deprecated import. Thanks @elmaghallawy!
2020-08-21 02:29:45 +00:00
### RQ 1.5.1 (2020-08-21)
* Fixes for Redis server version parsing. Thanks @selwin!
* Retries can now be set through @job decorator. Thanks @nerok!
* Log messages below logging.ERROR is now sent to stdout. Thanks @selwin!
* Better logger name for RQScheduler. Thanks @atainter!
* Better handling of exceptions thrown by horses. Thanks @theambient!
2020-07-26 10:49:09 +00:00
### RQ 1.5.0 (2020-07-26)
2020-09-10 01:19:53 +00:00
* Failed jobs can now be retried. Thanks @selwin!
2020-07-26 10:49:09 +00:00
* Fixed scheduler on Python > 3.8.0. Thanks @selwin!
* RQ is now aware of which version of Redis server it's running on. Thanks @aparcar!
* RQ now uses `hset()` on redis-py >= 3.5.0. Thanks @aparcar!
* Fix incorrect worker timeout calculation in SimpleWorker.execute_job(). Thanks @davidmurray!
* Make horse handling logic more robust. Thanks @wevsty!
2020-06-28 14:26:15 +00:00
### RQ 1.4.3 (2020-06-28)
* Added `job.get_position()` and `queue.get_job_position()`. Thanks @aparcar!
* Longer TTLs for worker keys to prevent them from expiring inside the worker lifecycle. Thanks @selwin!
* Long job args/kwargs are now truncated during logging. Thanks @JhonnyBn!
* `job.requeue()` now returns the modified job. Thanks @ericatkin!
2020-05-26 13:14:32 +00:00
### RQ 1.4.2 (2020-05-26)
* Reverted changes to `hmset` command which causes workers on Redis server < 4 to crash. Thanks @selwin!
* Merged in more groundwork to enable jobs with multiple dependencies. Thanks @thomasmatecki!
2020-05-16 10:46:55 +00:00
### RQ 1.4.1 (2020-05-16)
* Default serializer now uses `pickle.HIGHEST_PROTOCOL` for backward compatibility reasons. Thanks @bbayles!
* Avoid deprecation warnings on redis-py >= 3.5.0. Thanks @bbayles!
2020-05-13 01:10:29 +00:00
### RQ 1.4.0 (2020-05-13)
* Custom serializer is now supported. Thanks @solababs!
* `delay()` now accepts `job_id` argument. Thanks @grayshirt!
* Fixed a bug that may cause early termination of scheduled or requeued jobs. Thanks @rmartin48!
* When a job is scheduled, always add queue name to a set containing active RQ queue names. Thanks @mdawar!
* Added `--sentry-ca-certs` and `--sentry-debug` parameters to `rq worker` CLI. Thanks @kichawa!
* Jobs cleaned up by `StartedJobRegistry` are given an exception info. Thanks @selwin!
* Python 2.7 is no longer supported. Thanks @selwin!
2020-05-13 01:10:29 +00:00
2020-03-09 12:28:30 +00:00
### RQ 1.3.0 (2020-03-09)
* Support for infinite job timeout. Thanks @theY4Kman!
* Added `__main__` file so you can now do `python -m rq.cli`. Thanks @bbayles!
* Fixes an issue that may cause zombie processes. Thanks @wevsty!
* `job_id` is now passed to logger during failed jobs. Thanks @smaccona!
* `queue.enqueue_at()` and `queue.enqueue_in()` now supports explicit `args` and `kwargs` function invocation. Thanks @selwin!
2020-01-31 14:43:00 +00:00
### RQ 1.2.2 (2020-01-31)
* `Job.fetch()` now properly handles unpickleable return values. Thanks @selwin!
2020-01-31 08:38:44 +00:00
### RQ 1.2.1 (2020-01-31)
* `enqueue_at()` and `enqueue_in()` now sets job status to `scheduled`. Thanks @coolhacker170597!
* Failed jobs data are now automatically expired by Redis. Thanks @selwin!
* Fixes `RQScheduler` logging configuration. Thanks @FlorianPerucki!
Job scheduling (#1163) * First RQScheduler prototype * WIP job scheduling * Fixed Python 2.7 tests * Added ScheduledJobRegistry.get_scheduled_time(job) * WIP on scheduler's threading mechanism * Fixed test errors * Changed scheduler.acquire_locks() to instance method * Added scheduler.prepare_registries() * Somewhat working implementation of RQ scheduler * Only call stop_scheduler if there's a scheduler present * Use OSError rather than ProcessLookupError for PyPy compatibility * Added `auto_start` argument to scheduler.acquire_locks() * Make RQScheduler play better with timezone * Fixed test error * Added --with-scheduler flag to rq worker CLI * Fix tests on Python 2.x * More Python 2 fixes * Only call `scheduler.start` if worker is run in non burst mode * Fixed an issue where running worker with scheduler would fail sometimes * Make `worker.stop_scheduler()` more resilient to errors * worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks * Scheduler can now work with worker in both burst and non burst mode * Fixed scheduler logging message * Always log scheduler errors when running * Improve scheduler error logging message * Removed testing code * Scheduler should periodically try to acquire locks for other queues it doesn't have * Added tests for scheduler.should_reacquire_locks * Added queue.enqueue_in() * Fixes queue.enqueue_in() in Python 2.7 * First stab at documenting job scheduling * Remove unused methods * Remove Python 2.6 logging compatibility code * Remove more unused imports * Added convenience methods to access job registries from queue * Added test for worker.run_maintenance_tasks() * Simplify worker.queue_names() and worker.queue_keys() * Updated changelog to mention RQ's new job scheduling mechanism.
2020-01-04 03:14:52 +00:00
### RQ 1.2.0 (2020-01-04)
* This release also contains an alpha version of RQ's builtin job scheduling mechanism. Thanks @selwin!
2019-12-16 10:56:05 +00:00
* Various internal API changes in preparation to support multiple job dependencies. Thanks @thomasmatecki!
* `--verbose` or `--quiet` CLI arguments should override `--logging-level`. Thanks @zyt312074545!
* Fixes a bug in `rq info` where it doesn't show workers for empty queues. Thanks @zyt312074545!
* Fixed `queue.enqueue_dependents()` on custom `Queue` classes. Thanks @van-ess0!
* `RQ` and Python versions are now stored in job metadata. Thanks @eoranged!
* Added `failure_ttl` argument to job decorator. Thanks @pax0r!
2019-07-20 10:29:52 +00:00
### RQ 1.1.0 (2019-07-20)
- Added `max_jobs` to `Worker.work` and `--max-jobs` to `rq worker` CLI. Thanks @perobertson!
- Passing `--disable-job-desc-logging` to `rq worker` now does what it's supposed to do. Thanks @janierdavila!
- `StartedJobRegistry` now properly handles jobs with infinite timeout. Thanks @macintoshpie!
- `rq info` CLI command now cleans up registries when it first runs. Thanks @selwin!
- Replaced the use of `procname` with `setproctitle`. Thanks @j178!
2019-04-06 02:29:53 +00:00
### 1.0 (2019-04-06)
RQ v1.0! (#1059) * Added FailedJobRegistry. * Added job.failure_ttl. * queue.enqueue() now supports failure_ttl * Added registry.get_queue(). * FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL. * StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry. * Failed jobs are now added to FailedJobRegistry. * Added FailedJobRegistry.requeue() * Document the new `FailedJobRegistry` and changes in custom exception handler behavior. * Added worker.disable_default_exception_handler. * Document --disable-default-exception-handler option. * Deleted worker.failed_queue. * Deleted "move_to_failed_queue" exception handler. * StartedJobRegistry should no longer move jobs to FailedQueue. * Deleted requeue_job * Fixed test error. * Make requeue cli command work with FailedJobRegistry * Added .pytest_cache to gitignore. * Custom exception handlers are no longer run in reverse * Restored requeue_job function * Removed get_failed_queue * Deleted FailedQueue * Updated changelog. * Document `failure_ttl` * Updated docs. * Remove job.status * Fixed typo in test_registry.py * Replaced _pipeline() with pipeline() * FailedJobRegistry no longer fails on redis-py>=3 * Fixes test_clean_registries * Worker names are now randomized * Added a note about random worker names in CHANGES.md * Worker will now stop working when encountering an unhandled exception. * Worker should reraise SystemExit on cold shutdowns * Added anchor.js to docs * Support for Sentry-SDK (#1045) * Updated RQ to support sentry-sdk * Document Sentry integration * Install sentry-sdk before running tests * Improved rq info CLI command to be more efficient when displaying lar… (#1046) * Improved rq info CLI command to be more efficient when displaying large number of workers * Fixed an rq info --by-queue bug * Fixed worker.total_working_time bug (#1047) * queue.enqueue() no longer accepts `timeout` argument (#1055) * Clean worker registry (#1056) * queue.enqueue() no longer accepts `timeout` argument * Added clean_worker_registry() * Show worker hostname and PID on cli (#1058) * Show worker hostname and PID on cli * Improve test coverage * Remove Redis version check when SSL is used * Bump version to 1.0 * Removed pytest_cache/README.md * Changed worker logging to use exc_info=True * Removed unused queue.dequeue() * Fixed typo in CHANGES.md * setup_loghandlers() should always call logger.setLevel() if specified
2019-03-30 02:13:56 +00:00
Backward incompatible changes:
- `job.status` has been removed. Use `job.get_status()` and `job.set_status()` instead. Thanks @selwin!
- `FailedQueue` has been replaced with `FailedJobRegistry`:
* `get_failed_queue()` function has been removed. Please use `FailedJobRegistry(queue=queue)` instead.
* `move_to_failed_queue()` has been removed.
* RQ now provides a mechanism to automatically cleanup failed jobs. By default, failed jobs are kept for 1 year.
* Thanks @selwin!
- RQ's custom job exception handling mechanism has also changed slightly:
* RQ's default exception handling mechanism (moving jobs to `FailedJobRegistry`) can be disabled by doing `Worker(disable_default_exception_handler=True)`.
* Custom exception handlers are no longer executed in reverse order.
* Thanks @selwin!
- `Worker` names are now randomized. Thanks @selwin!
- `timeout` argument on `queue.enqueue()` has been deprecated in favor of `job_timeout`. Thanks @selwin!
- Sentry integration has been reworked:
* RQ now uses the new [sentry-sdk](https://pypi.org/project/sentry-sdk/) in place of the deprecated [Raven](https://pypi.org/project/raven/) library
* RQ will look for the more explicit `RQ_SENTRY_DSN` environment variable instead of `SENTRY_DSN` before instantiating Sentry integration
* Thanks @selwin!
- Fixed `Worker.total_working_time` accounting bug. Thanks @selwin!
2018-12-11 00:06:45 +00:00
### 0.13.0 (2018-12-11)
- Compatibility with Redis 3.0. Thanks @dash-rai!
- Added `job_timeout` argument to `queue.enqueue()`. This argument will eventually replace `timeout` argument. Thanks @selwin!
- Added `job_id` argument to `BaseDeathPenalty` class. Thanks @loopbio!
- Fixed a bug which causes long running jobs to timeout under `SimpleWorker`. Thanks @selwin!
- You can now override worker's name from config file. Thanks @houqp!
- Horses will now return exit code 1 if they don't terminate properly (e.g when Redis connection is lost). Thanks @selwin!
- Added `date_format` and `log_format` arguments to `Worker` and `rq worker` CLI. Thanks @shikharsg!
2018-07-14 01:00:38 +00:00
### 0.12.0 (2018-07-14)
- Added support for Python 3.7. Since `async` is a keyword in Python 3.7,
`Queue(async=False)` has been changed to `Queue(is_async=False)`. The `async`
keyword argument will still work, but raises a `DeprecationWarning`. Thanks @dchevell!
2018-12-11 00:06:45 +00:00
2018-07-14 01:00:38 +00:00
### 0.11.0 (2018-06-01)
2018-06-01 04:22:09 +00:00
- `Worker` now periodically sends heartbeats and checks whether child process is still alive while performing long running jobs. Thanks @Kriechi!
- `Job.create` now accepts `timeout` in string format (e.g `1h`). Thanks @theodesp!
- `worker.main_work_horse()` should exit with return code `0` even if job execution fails. Thanks @selwin!
- `job.delete(delete_dependents=True)` will delete job along with its dependents. Thanks @olingerc!
- Other minor fixes and documentation updates.
2018-12-11 00:06:45 +00:00
2018-01-02 05:59:02 +00:00
### 0.10.0
- `@job` decorator now accepts `description`, `meta`, `at_front` and `depends_on` kwargs. Thanks @jlucas91 and @nlyubchich!
- Added the capability to fetch workers by queue using `Worker.all(queue=queue)` and `Worker.count(queue=queue)`.
- Improved RQ's default logging configuration. Thanks @samuelcolvin!
- `job.data` and `job.exc_info` are now stored in compressed format in Redis.
2018-12-11 00:06:45 +00:00
2017-11-21 09:53:44 +00:00
### 0.9.2
- Fixed an issue where `worker.refresh()` may fail when `birth_date` is not set. Thanks @vanife!
2018-12-11 00:06:45 +00:00
2017-11-03 06:00:40 +00:00
### 0.9.1
- Fixed an issue where `worker.refresh()` may fail when upgrading from previous versions of RQ.
2018-12-11 00:06:45 +00:00
2017-10-28 10:37:26 +00:00
### 0.9.0
- `Worker` statistics! `Worker` now keeps track of `last_heartbeat`, `successful_job_count`, `failed_job_count` and `total_working_time`. Thanks @selwin!
- `Worker` now sends heartbeat during suspension check. Thanks @theodesp!
- Added `queue.delete()` method to delete `Queue` objects entirely from Redis. Thanks @theodesp!
- More robust exception string decoding. Thanks @stylight!
- Added `--logging-level` option to command line scripts. Thanks @jiajunhuang!
- Added millisecond precision to job timestamps. Thanks @samuelcolvin!
- Python 2.6 is no longer supported. Thanks @samuelcolvin!
2018-12-11 00:06:45 +00:00
2017-09-05 03:57:52 +00:00
### 0.8.2
- Fixed an issue where `job.save()` may fail with unpickleable return value.
2017-07-28 08:52:48 +00:00
2018-12-11 00:06:45 +00:00
### 0.8.1
2017-07-28 08:52:48 +00:00
- Replace `job.id` with `Job` instance in local `_job_stack `. Thanks @katichev!
- `job.save()` no longer implicitly calls `job.cleanup()`. Thanks @katichev!
- Properly catch `StopRequested` `worker.heartbeat()`. Thanks @fate0!
- You can now pass in timeout in days. Thanks @yaniv-g!
- The core logic of sending job to `FailedQueue` has been moved to `rq.handlers.move_to_failed_queue`. Thanks @yaniv-g!
- RQ cli commands now accept `--path` parameter. Thanks @kirill and @sjtbham!
- Make `job.dependency` slightly more efficient. Thanks @liangsijian!
- `FailedQueue` now returns jobs with the correct class. Thanks @amjith!
2018-12-11 00:06:45 +00:00
### 0.8.0
- Refactored APIs to allow custom `Connection`, `Job`, `Worker` and `Queue` classes via CLI. Thanks @jezdez!
- `job.delete()` now properly cleans itself from job registries. Thanks @selwin!
- `Worker` should no longer overwrite `job.meta`. Thanks @WeatherGod!
2017-05-01 10:09:31 +00:00
- `job.save_meta()` can now be used to persist custom job data. Thanks @katichev!
- Added Redis Sentinel support. Thanks @strawposter!
- Make `Worker.find_by_key()` more efficient. Thanks @selwin!
- You can now specify job `timeout` using strings such as `queue.enqueue(foo, timeout='1m')`. Thanks @luojiebin!
- Better unicode handling. Thanks @myme5261314 and @jaywink!
- Sentry should default to HTTP transport. Thanks @Atala!
- Improve `HerokuWorker` termination logic. Thanks @samuelcolvin!
2016-12-04 00:58:01 +00:00
### 0.7.1
- Fixes a bug that prevents fetching jobs from `FailedQueue` (#765). Thanks @jsurloppe!
- Fixes race condition when enqueueing jobs with dependency (#742). Thanks @th3hamm0r!
- Skip a test that requires Linux signals on MacOS (#763). Thanks @jezdez!
2017-11-21 09:53:44 +00:00
- `enqueue_job` should use Redis pipeline when available (#761). Thanks mtdewulf!
2016-12-04 00:58:01 +00:00
2016-06-21 07:10:08 +00:00
2018-12-11 00:06:45 +00:00
### 0.7.0
2016-11-18 07:26:35 +00:00
- Better support for Heroku workers (#584, #715)
- Support for connecting using a custom connection class (#741)
- Fix: connection stack in default worker (#479, #641)
- Fix: `fetch_job` now checks that a job requested actually comes from the
intended queue (#728, #733)
- Fix: Properly raise exception if a job dependency does not exist (#747)
- Fix: Job status not updated when horse dies unexpectedly (#710)
- Fix: `request_force_stop_sigrtmin` failing for Python 3 (#727)
- Fix `Job.cancel()` method on failed queue (#707)
- Python 3.5 compatibility improvements (#729)
- Improved signal name lookup (#722)
2016-06-21 07:10:08 +00:00
2016-05-08 14:31:55 +00:00
### 0.6.0
- Jobs that depend on job with result_ttl == 0 are now properly enqueued.
- `cancel_job` now works properly. Thanks @jlopex!
- Jobs that execute successfully now no longer tries to remove itself from queue. Thanks @amyangfei!
- Worker now properly logs Falsy return values. Thanks @liorsbg!
- `Worker.work()` now accepts `logging_level` argument. Thanks @jlopex!
- Logging related fixes by @redbaron4 and @butla!
- `@job` decorator now accepts `ttl` argument. Thanks @javimb!
- `Worker.__init__` now accepts `queue_class` keyword argument. Thanks @antoineleclair!
- `Worker` now saves warm shutdown time. You can access this property from `worker.shutdown_requested_date`. Thanks @olingerc!
- Synchronous queues now properly sets completed job status as finished. Thanks @ecarreras!
- `Worker` now correctly deletes `current_job_id` after failed job execution. Thanks @olingerc!
- `Job.create()` and `queue.enqueue_call()` now accepts `meta` argument. Thanks @tornstrom!
- Added `job.started_at` property. Thanks @samuelcolvin!
- Cleaned up the implementation of `job.cancel()` and `job.delete()`. Thanks @glaslos!
- `Worker.execute_job()` now exports `RQ_WORKER_ID` and `RQ_JOB_ID` to OS environment variables. Thanks @mgk!
- `rqinfo` now accepts `--config` option. Thanks @kfrendrich!
- `Worker` class now has `request_force_stop()` and `request_stop()` methods that can be overridden by custom worker classes. Thanks @samuelcolvin!
- Other minor fixes by @VicarEscaped, @kampfschlaefer, @ccurvey, @zfz, @antoineleclair,
@orangain, @nicksnell, @SkyLothar, @ahxxm and @horida.
2015-10-08 11:15:08 +00:00
### 0.5.6
- Job results are now logged on `DEBUG` level. Thanks @tbaugis!
- Modified `patch_connection` so Redis connection can be easily mocked
- Customer exception handlers are now called if Redis connection is lost. Thanks @jlopex!
- Jobs can now depend on jobs in a different queue. Thanks @jlopex!
2015-08-25 07:08:36 +00:00
2018-12-11 00:06:45 +00:00
### 0.5.5 (2015-08-25)
2015-08-25 07:08:36 +00:00
- Add support for `--exception-handler` command line flag
- Fix compatibility with click>=5.0
- Fix maximum recursion depth problem for very large queues that contain jobs
that all fail
2015-07-08 07:11:05 +00:00
### 0.5.4
(July 8th, 2015)
- Fix compatibility with raven>=5.4.0
2015-06-01 04:00:36 +00:00
### 0.5.3
2015-06-03 08:55:16 +00:00
(June 3rd, 2015)
2015-06-01 04:00:36 +00:00
- Better API for instantiating Workers. Thanks @RyanMTB!
2015-06-03 08:55:16 +00:00
- Better support for unicode kwargs. Thanks @nealtodd and @brownstein!
- Workers now automatically cleans up job registries every hour
- Jobs in `FailedQueue` now have their statuses set properly
- `enqueue_call()` no longer ignores `ttl`. Thanks @mbodock!
- Improved logging. Thanks @trevorprater!
2015-06-01 04:00:36 +00:00
2015-04-14 07:10:35 +00:00
### 0.5.2
(April 14th, 2015)
- Support SSL connection to Redis (requires redis-py>=2.10)
- Fix to prevent deep call stacks with large queues
2015-03-09 08:18:34 +00:00
### 0.5.1
(March 9th, 2015)
- Resolve performance issue when queues contain many jobs
- Restore the ability to specify connection params in config
- Record `birth_date` and `death_date` on Worker
- Add support for SSL URLs in Redis (and `REDIS_SSL` config option)
- Fix encoding issues with non-ASCII characters in function arguments
- Fix Redis transaction management issue with job dependencies
2015-01-30 07:39:42 +00:00
### 0.5.0
(Jan 30th, 2015)
- RQ workers can now be paused and resumed using `rq suspend` and
`rq resume` commands. Thanks Jonathan Tushman!
- Jobs that are being performed are now stored in `StartedJobRegistry`
for monitoring purposes. This also prevents currently active jobs from
being orphaned/lost in the case of hard shutdowns.
- You can now monitor finished jobs by checking `FinishedJobRegistry`.
Thanks Nic Cope for helping!
- Jobs with unmet dependencies are now created with `deferred` as their
status. You can monitor deferred jobs by checking `DeferredJobRegistry`.
- It is now possible to enqueue a job at the beginning of queue using
`queue.enqueue(func, at_front=True)`. Thanks Travis Johnson!
- Command line scripts have all been refactored to use `click`. Thanks Lyon Zhang!
- Added a new `SimpleWorker` that does not fork when executing jobs.
Useful for testing purposes. Thanks Cal Leeming!
- Added `--queue-class` and `--job-class` arguments to `rqworker` script.
Thanks David Bonner!
- Many other minor bug fixes and enhancements.
2014-05-18 06:14:07 +00:00
### 0.4.6
2014-05-21 08:08:47 +00:00
(May 21st, 2014)
2014-05-18 06:14:07 +00:00
2014-05-21 08:08:47 +00:00
- Raise a warning when RQ workers are used with Sentry DSNs using
asynchronous transports. Thanks Wei, Selwin & Toms!
2014-05-18 06:14:07 +00:00
2014-05-08 11:56:57 +00:00
### 0.4.5
(May 8th, 2014)
- Fix where rqworker broke on Python 2.6. Thanks, Marko!
2014-05-07 18:49:48 +00:00
### 0.4.4
(May 7th, 2014)
- Properly declare redis dependency.
- Fix a NameError regression that was introduced in 0.4.3.
2014-05-06 13:58:35 +00:00
### 0.4.3
(May 6th, 2014)
- Make job and queue classes overridable. Thanks, Marko!
- Don't require connection for @job decorator at definition time. Thanks, Sasha!
- Syntactic code cleanup.
2014-04-28 06:31:09 +00:00
### 0.4.2
(April 28th, 2014)
- Add missing depends_on kwarg to @job decorator. Thanks, Sasha!
2014-04-22 20:21:37 +00:00
### 0.4.1
(April 22nd, 2014)
- Fix bug where RQ 0.4 workers could not unpickle/process jobs from RQ < 0.4.
2013-09-03 12:18:16 +00:00
### 0.4.0
2014-04-22 14:10:19 +00:00
(April 22nd, 2014)
2013-08-26 08:21:48 +00:00
- Emptying the failed queue from the command line is now as simple as running
`rqinfo -X` or `rqinfo --empty-failed-queue`.
2014-01-06 23:23:36 +00:00
- Job data is unpickled lazily. Thanks, Malthe!
- Removed dependency on the `times` library. Thanks, Malthe!
2013-09-03 12:18:16 +00:00
- Job dependencies! Thanks, Selwin.
- Custom worker classes, via the `--worker-class=path.to.MyClass` command line
argument. Thanks, Selwin.
2013-09-04 22:29:40 +00:00
- `Queue.all()` and `rqinfo` now report empty queues, too. Thanks, Rob!
- Fixed a performance issue in `Queue.all()` when issued in large Redis DBs.
Thanks, Rob!
2013-09-05 07:25:00 +00:00
- Birth and death dates are now stored as proper datetimes, not timestamps.
2013-08-26 08:21:48 +00:00
- Ability to provide a custom job description (instead of using the default
function invocation hint). Thanks, İbrahim.
2013-09-05 19:05:03 +00:00
- Fix: temporary key for the compact queue is now randomly generated, which
should avoid name clashes for concurrent compact actions.
- Fix: `Queue.empty()` now correctly deletes job hashes from Redis.
2013-09-02 20:54:49 +00:00
2013-08-26 08:21:48 +00:00
2013-12-17 11:28:57 +00:00
### 0.3.13
(December 17th, 2013)
- Bug fix where the worker crashes on jobs that have their timeout explicitly
removed. Thanks for reporting, @algrs.
### 0.3.12
(December 16th, 2013)
- Bug fix where a worker could time out before the job was done, removing it
from any monitor overviews (#288).
2013-08-23 13:11:58 +00:00
### 0.3.11
(August 23th, 2013)
- Some more fixes in command line scripts for Python 3
2013-08-20 10:06:43 +00:00
### 0.3.10
(August 20th, 2013)
- Bug fix in setup.py
2013-08-20 09:17:05 +00:00
### 0.3.9
(August 20th, 2013)
- Python 3 compatibility (Thanks, Alex!)
- Minor bug fix where Sentry would break when func cannot be imported
### 0.3.8
2013-06-17 06:48:57 +00:00
(June 17th, 2013)
2013-02-18 07:10:47 +00:00
2013-04-19 08:55:17 +00:00
- `rqworker` and `rqinfo` have a `--url` argument to connect to a Redis url.
2013-03-16 14:08:01 +00:00
- `rqworker` and `rqinfo` have a `--socket` option to connect to a Redis server
through a Unix socket.
- `rqworker` reads `SENTRY_DSN` from the environment, unless specifically
provided on the command line.
2013-02-18 07:10:47 +00:00
2013-04-19 08:55:17 +00:00
- `Queue` has a new API that supports paging `get_jobs(3, 7)`, which will
return at most 7 jobs, starting from the 3rd.
2013-02-18 07:10:47 +00:00
### 0.3.7
(February 26th, 2013)
- Fixed bug where workers would not execute builtin functions properly.
2013-02-06 22:28:03 +00:00
### 0.3.6
2013-02-18 07:09:06 +00:00
(February 18th, 2013)
2013-02-06 22:28:03 +00:00
2013-02-15 07:53:42 +00:00
- Worker registrations now expire. This should prevent `rqinfo` from reporting
about ghosted workers. (Thanks, @yaniv-aknin!)
2013-02-06 22:28:03 +00:00
2013-02-18 06:59:08 +00:00
- `rqworker` will automatically clean up ghosted worker registrations from
pre-0.3.6 runs.
- `rqworker` grew a `-q` flag, to be more silent (only warnings/errors are shown)
2013-02-06 22:28:03 +00:00
2013-01-23 19:52:20 +00:00
### 0.3.5
2013-02-06 22:26:52 +00:00
(February 6th, 2013)
2013-01-23 19:52:20 +00:00
2013-01-23 21:21:22 +00:00
- `ended_at` is now recorded for normally finished jobs, too. (Previously only
for failed jobs.)
2013-01-23 19:52:20 +00:00
- Adds support for both `Redis` and `StrictRedis` connection types
- Makes `StrictRedis` the default connection type if none is explicitly provided
2013-01-23 19:52:20 +00:00
2013-01-18 15:23:43 +00:00
### 0.3.4
2013-01-23 19:48:49 +00:00
(January 23rd, 2013)
2013-01-18 15:23:43 +00:00
2013-01-23 19:48:49 +00:00
- Restore compatibility with Python 2.6.
2013-01-18 15:23:43 +00:00
2012-09-03 06:15:47 +00:00
### 0.3.3
2013-01-18 14:51:17 +00:00
(January 18th, 2013)
2012-09-03 06:15:47 +00:00
2013-01-18 14:12:49 +00:00
- Fix bug where work was lost due to silently ignored unpickle errors.
2012-09-03 11:54:39 +00:00
- Jobs can now access the current `Job` instance from within. Relevant
documentation [here](http://python-rq.org/docs/jobs/).
- Custom properties can be set by modifying the `job.meta` dict. Relevant
documentation [here](http://python-rq.org/docs/jobs/).
2012-09-02 09:30:45 +00:00
2012-09-03 11:53:52 +00:00
- Custom properties can be set by modifying the `job.meta` dict. Relevant
documentation [here](http://python-rq.org/docs/jobs/).
2012-09-03 06:15:47 +00:00
2012-09-05 06:37:40 +00:00
- `rqworker` now has an optional `--password` flag.
2013-01-18 14:48:27 +00:00
- Remove `logbook` dependency (in favor of `logging`)
2012-09-03 06:15:47 +00:00
2012-09-02 09:30:45 +00:00
### 0.3.2
2012-09-03 06:13:22 +00:00
(September 3rd, 2012)
- Fixes broken `rqinfo` command.
- Improve compatibility with Python < 2.7.
2012-09-02 09:30:45 +00:00
2012-08-07 09:09:08 +00:00
### 0.3.1
2012-08-30 19:37:29 +00:00
(August 30th, 2012)
2012-08-07 09:09:08 +00:00
- `.enqueue()` now takes a `result_ttl` keyword argument that can be used to
change the expiration time of results.
2012-08-23 07:54:27 +00:00
- Queue constructor now takes an optional `async=False` argument to bypass the
worker (for testing purposes).
2012-08-27 12:22:02 +00:00
- Jobs now carry status information. To get job status information, like
whether a job is queued, finished, or failed, use the property `status`, or
one of the new boolean accessor properties `is_queued`, `is_finished` or
`is_failed`.
- Jobs return values are always stored explicitly, even if they have to
explicit return value or return `None` (with given TTL of course). This
makes it possible to distinguish between a job that explicitly returned
`None` and a job that isn't finished yet (see `status` property).
2012-08-29 11:06:41 +00:00
- Custom exception handlers can now be configured in addition to, or to fully
2012-08-29 11:11:01 +00:00
replace, moving failed jobs to the failed queue. Relevant documentation
[here](http://python-rq.org/docs/exceptions/) and
[here](http://python-rq.org/patterns/sentry/).
2012-08-29 11:06:41 +00:00
2012-08-29 12:51:26 +00:00
- `rqworker` now supports passing in configuration files instead of the
many command line options: `rqworker -c settings` will source
`settings.py`.
2012-08-29 12:57:40 +00:00
- `rqworker` now supports one-flag setup to enable Sentry as its exception
handler: `rqworker --sentry-dsn="http://public:secret@example.com/1"`
Alternatively, you can use a settings file and configure `SENTRY_DSN
= 'http://public:secret@example.com/1'` instead.
2012-08-07 09:09:08 +00:00
2012-07-23 10:01:35 +00:00
### 0.3.0
(August 5th, 2012)
- Reliability improvements
- Warm shutdown now exits immediately when Ctrl+C is pressed and worker is idle
- Worker does not leak worker registrations anymore when stopped gracefully
2012-07-20 12:50:23 +00:00
2012-07-24 09:28:18 +00:00
- `.enqueue()` does not consume the `timeout` kwarg anymore. Instead, to pass
RQ a timeout value while enqueueing a function, use the explicit invocation
instead:
2012-08-02 12:52:12 +00:00
```python
2012-07-24 09:28:18 +00:00
q.enqueue(do_something, args=(1, 2), kwargs={'a': 1}, timeout=30)
2012-08-02 12:52:12 +00:00
```
2012-07-24 09:28:18 +00:00
- Add a `@job` decorator, which can be used to do Celery-style delayed
invocations:
2012-08-02 12:52:12 +00:00
```python
from redis import StrictRedis
2012-07-24 09:28:18 +00:00
from rq.decorators import job
2012-07-25 07:11:13 +00:00
# Connect to Redis
redis = StrictRedis()
2012-07-25 07:11:13 +00:00
@job('high', timeout=10, connection=redis)
2012-07-24 09:28:18 +00:00
def some_work(x, y):
return x + y
2012-08-02 12:52:12 +00:00
```
2012-07-24 09:28:18 +00:00
2012-07-25 07:11:13 +00:00
Then, in another module, you can call `some_work`:
2012-08-02 12:52:12 +00:00
```python
2012-07-25 07:11:13 +00:00
from foo.bar import some_work
2012-07-24 09:28:18 +00:00
some_work.delay(2, 3)
2012-08-02 12:52:12 +00:00
```
2012-07-20 12:50:23 +00:00
2012-08-01 11:54:05 +00:00
### 0.2.2
(August 1st, 2012)
- Fix bug where return values that couldn't be pickled crashed the worker
2012-07-20 12:49:18 +00:00
### 0.2.1
(July 20th, 2012)
2012-07-18 12:43:28 +00:00
- Fix important bug where result data wasn't restored from Redis correctly
(affected non-string results only).
2012-07-18 12:43:28 +00:00
### 0.2.0
(July 18th, 2012)
- `q.enqueue()` accepts instance methods now, too. Objects will be pickle'd
along with the instance method, so beware.
- `q.enqueue()` accepts string specification of functions now, too. Example:
`q.enqueue("my.math.lib.fibonacci", 5)`. Useful if the worker and the
submitter of work don't share code bases.
- Job can be assigned custom attrs and they will be pickle'd along with the
rest of the job's attrs. Can be used when writing RQ extensions.
- Workers can now accept explicit connections, like Queues.
- Various bug fixes.
2012-05-20 14:09:57 +00:00
2012-05-15 06:38:10 +00:00
### 0.1.2
(May 15, 2012)
- Fix broken PyPI deployment.
2012-05-14 06:41:56 +00:00
### 0.1.1
(May 14, 2012)
- Thread-safety by using context locals
- Register scripts as console_scripts, for better portability
- Various bugfixes.
### 0.1.0:
(March 28, 2012)
- Initially released version.