* Improve docstrings on `connections`
* Enhanced Job methods docstrings & Serialization Protocol
This adds docstrings to all Job methods in a standard format.
It also implements a `serializer` protocol.
* Excludes `Protocol` (keeping compatibility with < 3.8)
* Add docstrings & type annotation to the `job` decorator
* Docstrings for the `defaults` vars
* Add deprecation warning to Connection context manager
* Fix Types
* Moves the `compact` helper to `utils`
There is a helper funcion that excludes `None` values
from a list. This was being declared both in the Queue
and in the worker. This centralizes this helper in the `utils`
Importing
* Fix Type Annotation
* Remove unused code from compat module
* Remove unused dictconfig
* Remove total_ordering compat layer
* Remove compatibility layer
This completely removes the compat module. It moves utilities
functions (`as_text` and `decode_redis_hash`) to the `utils`
module, is eliminates the use of the proxies `text_type` and
`string_types`, using the `str` construct directly.
* Remove compat module
Finishes the cleaning of the compatibility module.
The last function being the `is_python_version` function
which was being used internally.
* Fix old import
* Fix Imports
* Remove Dummy (Force GH Actions)
* Fix Imports
* Organize Imports
* Add debug logs to dequeue
There were reports where a worker wouldn't fetch new jobs.
Since one possible cause is the infinite hanging of the `BLPOP` command,
this adds a couple of logs surrounding the `dequeue_any` function,
that interacts with Redis using `BLPOP`,
* Update worker.py
Fix Typo
* More logs
* Logs
* Update worker.py
* Add debug logs to the queue
* Reviewed logs
* Remove queue count before job push
* New log wording
* Remove logs
* Allow for configurable scheduler fallback period
This adds a parameter to the Scheduler called
`fallback_period`, which determines the period
before work fallsback to a new scheduler.
* Update defaults.py
Reduce Scheduler Fallback Period to 120 seconds.
* Update scheduler.py
Remove fallback period parameter.
Dummy tasks were probably used for load testing
in the past, but currently are not being imported from anywhere.
Also, similar task utilities are already present on the tests fixtures,
making the `dummy` module irrelevant.
* Persist worker_name after job is finished
Persisting the worker_name on the job object in Redis would allow for debugging and
analyzing logs from the worker
* Remove redundent job.save() method call
* Remove check for null worker
Now that worker name is persisted after job finishes or fails,
no need to assert that worker name is None
* Change github runner to Ubuntu 20.04
* Change github runner to Ubuntu 20.04
* WIP job results
* Result can now be saved
* Successfully saved and restored result
* result.save() should accept pipeline
* Successful results are saved
* Failures are now saved properly too.
* Added test for Result.get_latest()
* Checkpoint
* Got Result.all() to work
* Added Result.count(), Result.delete()
* Backward compatibility for job.result and job.exc_info
* Added some typing
* More typing stuff
* Fixed typing in job.py
* More typing updates
* Only keep the last 10 results
* Documented job.results()
* Got results test to pass
* Don't run test_results.py on Redis server < 5.0
* Fixed mock import on some Python versions
* Remove Redis 3 from test matrix
* Jobs should never use the new Result implementation if server is < 5.0
* Results should only be created is Redis stream is supported.
* Added back Redis 3 to test matrix
* Fixed job.supports_redis_streams
* Fixed worker test
* Updated docs.
* Add feature to enqueue dependents at the front of queues
* Add documentation for the Dependency(enqueue_at_front=...) parameter
* docs: Add `enqueue_at_front` to list of parameters for Dependency
* test: Update dependency test to not rely on Redis ordering
* refactor: Save enqueue_at_front boolean in job.meta instead of separate instance attr
* fix: Made enqueue_at_front an instance attribute instead of putting it inside meta
* doc: default queues order is a priority order
In introduction, we say that queue order matters, but we are not very specific.
Added a paragraph to explicit that.
* doc: fix typo
* Fix job.dependencies_are_met() if dependency is canceled
* Slightly better test coverage on dependencies_are_met()
* Fixed job.cancel(enqueue_dependent=True)
* Move common flake8 options into config file
Currently --max-line-length being specified in two places. Just use the
existing value in the config file as the source of truth.
Move --count and --statistics to config file as well.
* Fix some lints
The on_success and on_failure callbacks are currently supported in
Queue.enqueue but not Queue.enqueue_many, despite documentary indicating
the contrary.
This commit brings the bulk enqueuing API back into parity.
Fixes#1680
* added Dependency class with allow_failures
* Requested changes
* Check type before setting `job.dependency_allow_fail` within `Job.create`
* Set `job.dependency_allow_fail` within `Job.create`
* Added test to ensure persistence of `dependency_allow_fail`
* Removed typing and allow mixed list of ints and Job objects
* Convert dependency_allow_fail boolean to integer during serialization to avoid redis DataError
* Updated `test_multiple_dependencies_are_accepted_and_persisted` test to include `Dependency` cases
* Adding placeholder test to test actual behavior of new `Dependency` usage in `depends_on`
* Updated `test_job_dependency` to include cases using `Dependency`
* Added dependency_allow_fail logic to `Job.restore`
* Renamed `dependency_allow_fail` to a simpler `allow_failure`
* Update docs to add section about the new `Dependency` class and use-case
* Updated `Job.dependencies_are_met` logic to take `FAILED` and `STOPPED` jobs into account when `allow_failure=True`
* Updated `test_job_dependency` test. Still failing with `Dependency` case.
* Fix `allow_failure` type coercion in `Job.restore`
* Re-arrange tests, so default `Dependency.allow_failure` is before explicit `allow_failure=True`
* Fixed Dependency, so it works correctly when allow_failure=True
* Attempt to execute pipeline prior to queueing a failed job's dependents. test_create_and_cancel_job_enqueue_dependents_in_registry test now passes.
* Added `Depedency` test utilizing multiple dependencies
* Removed irrelevant on_success and on_failure keyword arguments in example
* Replaced use of long_running_job
* Add test to verify `Dependency.jobs` contraints
* Suppress connection error in handle_job_failure
* test_dependencies have passed
* All tests pass if enqueue_dependents called without pipeline.watch()
* All tests now pass
* Removed print statements
* Cleanup Dependency implementation
* Renamed job.allow_failure to job.allow_dependency_failures
Co-authored-by: mattchan <mattchan@tencent.com>
Co-authored-by: Mike Hill <mhilluniversal@gmail.com>
There are small typos in:
- docs/docs/exceptions.md
- docs/docs/jobs.md
- rq/queue.py
- tests/fixtures.py
- tests/test_job.py
Fixes:
- Should read `slightly` rather than `slighty`.
- Should read `requeuing` rather than `requeueing`.
- Should read `implementers` rather than `implementors`.
- Should read `definition` rather than `defition`.
- Should read `canceled` rather than `canceld`.
Signed-off-by: Tim Gates <tim.gates@iress.com>
lowercase00