* Ensure that restore is atomic in redis transport
* Fix wrong variable
* Added integration tests
* Update unittests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix integration tests
* Fix flake8
* Fix queue name in int. tests
* Cosmetic clean up of integration tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Missing redis key containg set of queues bound to queue is caused
by removal all queues from the exchange. Hence, we should not raise an
exception but just return empty list of queues instead. This commit
fixes e.g. case publishing message against empty exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
exchange.declare(channel=conn.default_channel)
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
But it also fixes the case when last queue is unbound from exchange and
after publishing to this exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
queue1 = kombu.Queue('queue1', exchange=exchange, routing_key='queue1')
exchange.declare(channel=conn.default_channel)
queue1 = queue1.bind(channel=conn.default_channel)
queue1.declare()
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
queue1.delete()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/pyupgrade: v2.26.0 → v2.28.0](https://github.com/asottile/pyupgrade/compare/v2.26.0...v2.28.0)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Remove dependency to case
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix flake8 errors
* Remove unused code
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Add missing DEL, RPUSH, RPOP and SREM commands to the list of commands to prefix.
Also, this commit refactors the prefixing logic a bit to make it simpler.
* Introduce global key prefix for redis transport
Co-authored-by: Matus Valo <matusvalo@users.noreply.github.com>
* refactor: use a custom redis client
As per the suggestions, refactor the redis key prefixing to use a custom
redis client that prefixes the keys it uses.
The custom client implementation does not prefix every key by default as
the way of prefixing keys may differ for some redis commands, instead it
lists those keys that will be prefixed. In case of commands, where
multiple keys can be passed as an argument, the custom client defines
where the arg positions are starting and ending for the given command.
* test: fix unit tests by moving import statement
* fix: wrap redis.parse_response to remove key prefixes
Co-authored-by: Matus Valo <matusvalo@users.noreply.github.com>
* fix: typo
* fix: lint
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
Co-authored-by: Matus Valo <matusvalo@users.noreply.github.com>
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Previously, calling `reject` when `predefined_queues` was not configured would cause `AttributeError` to be raised from `_extract_backoff_policy_configuration_and_message`. That exception could crash the whole Celery worker and force it to exit early because `AttributeError` is not excepted in the nearby call stack.
* enable pre-commit
* use extend-ignore for flake8
* manual flake8 fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update kombu/__init__.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Move pytest.importorskip to run earlier
Currently pytest fails to collect tests when `azure` is not present. Moving the pytest.importorskip clause before `from kombu.transport import azureservicebus` fixes it.
```
==================================== ERRORS ====================================
__________ ERROR collecting t/unit/transport/test_azureservicebus.py ___________
ImportError while importing test module '/build/python-kombu/src/kombu-5.1.0/t/unit/transport/test_azureservicebus.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
t/unit/transport/test_azureservicebus.py:12: in <module>
from kombu.transport import azureservicebus
kombu/transport/azureservicebus.py:66: in <module>
import azure.core.exceptions
E ModuleNotFoundError: No module named 'azure'
```
* Add # noqa to make linter happy
* fixed character replace table according to the comment above - dots replaced by dashes, other punctuations replaced by underscores
* optimised with precalculated punctuation set - according to @thedrow suggestion
* queue name tests
* cleanup
* fix: non kombu json message decoding in SQS transport
* fix: non kombu json message decoding in SQS transport - add tests
Co-authored-by: Max Nikitenko <max.nikitenko@namecheap.com>
* Port of redis code improvements from prior revision
Add fakeredis requirement
* refine variable naming based on PR feedback.
* Remove python2 code style and remove unittest.Testcase from test_redis.py
* Code refactor of redis.py and test_redis.py
Co-authored-by: Matus Valo <matusvalo@gmail.com>
Use the SentinelManagedSSLConnection when SSL is enabled for the
transport. The redis-py project doesn't have a connection class for
SSL+Sentinel yet. So, create a class in redis.py to add that
functionality.
* Fix for the issue #1172
* Unit test for the fix relating to the issue #1172
* Fix for issue #1198: Celery crashes in cases where there aren’t enough available workers to start acting on “in-flight” messages in the SQS queue
* Fix for issue #1198: fixed lint issues
* Fix for issue #1198: added unit tests
Co-authored-by: inauros <inauros@copyright.com>
* Add support for health_check_interval option in broker_transport_options. #1114
* Added Keivn Fox to authors.
* Do not use default arg for pop() method.
* Add dedicated test case for configurable health_check
* Add test case for when health_check_interval is not supported by connection_class in redis.
* Fix Flake8
* Added more test cases for redis.
* Updated tests.
Matus Valo