* Redelivered messages should respect the original priority
* update restore message test to check priority
* flake8
* add integration tests
* also add integration test for mongodb
* flake8
* temporarily removing python 3.9 from CI due to unrelated failures
* Update .github/workflows/ci.yaml
---------
Co-authored-by: Tomer Nosrati <tomer.nosrati@gmail.com>
Co-authored-by: Asif Saif Uddin <auvipy@gmail.com>
The redis-py 4.5.2 changes the UnixDomainSocketConnection class so now
it inherits from AbstractConnection:
https://github.com/redis/redis-py/releases/tag/v4.5.2
This patch makes sure that the health_check_interval parameter is
checked for the __init__ method of the main class and also the bases, so
it doesn't fail with the newer version of redis-py.
* Support pymongo 4.x
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix problems detected by CI
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* allow getting recoverable_connection_errors without an active transport
* move redis transport errors to class
* move consul transport errors to class
* move etcd transport errors to class
* remove redis.Transport._get_errors and references in tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix flake8 errors
* add integration test for redis ConnectionError
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Ensure that restore is atomic in redis transport
* Fix wrong variable
* Added integration tests
* Update unittests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix integration tests
* Fix flake8
* Fix queue name in int. tests
* Cosmetic clean up of integration tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Missing redis key containg set of queues bound to queue is caused
by removal all queues from the exchange. Hence, we should not raise an
exception but just return empty list of queues instead. This commit
fixes e.g. case publishing message against empty exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
exchange.declare(channel=conn.default_channel)
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
But it also fixes the case when last queue is unbound from exchange and
after publishing to this exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
queue1 = kombu.Queue('queue1', exchange=exchange, routing_key='queue1')
exchange.declare(channel=conn.default_channel)
queue1 = queue1.bind(channel=conn.default_channel)
queue1.declare()
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
queue1.delete()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
* enable pre-commit
* use extend-ignore for flake8
* manual flake8 fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update kombu/__init__.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add transports based on Azure PaaS
This pull request adds two new transport implementations:
- `azurestoragequeues` is implemented on top of Azure Storage
Queues [1]. This offers a simple but scalable and low-cost PaaS
transport for Celery users in Azure. The transport is intended to be
used in conjunction with the Azure Block Blob Storage backend [2].
- `azureservicebus` is implemented on top of Azure Service Bus [3] and
offers PaaS support for more demanding Celery workloads in Azure. The
transport is intended to be used in conjunction with the Azure
CosmosDB backend [4].
This pull request was created together with @ankurokok, @dkisselev,
@evandropaula, @martinpeck and @michaelperel.
[1] https://azure.microsoft.com/en-us/services/storage/queues/
[2] https://github.com/celery/celery/pull/4685
[3] https://azure.microsoft.com/en-us/services/service-bus/
[4] https://github.com/celery/celery/pull/4720
* Exclude Azure transports from code coverage
There is test coverage for the transports but the tests require Azure
credentials to run (passed via environment variables) so codecov doesn't
exercise them.
* Remove env vars to configure transport
* Remove abbreviations
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Switch Boto2 to Boto3 for SQS messaging
* Fixed region support
* Add SQS FIFO queue support
* Add sensible defaults for message attributes
* Asynchronous support, plus boto3 for region endpoint lookups
* Clean up imports
* Fix Python 2 support
* Fix receive_message tests
* Reformat docstring
* boto3 import changes for CI
* skip tests if boto3 not installed
* skip tests if boto3 not installed
* flake8
* noboto
* ditching boto2. got queue URL fetching, async HTTP request generation and signing working.
* request signing working kinda
* async parsing of SQS message response more or less working
* botocore sqs dep
* ripping out more old boto2 stuff
* removing tests that are no longer valid with boto3/SQS
* fix boto3 dep, min version and no botocore
* no boto2 for test
* cleaning up some SQS tests. fixing header parsing of response to msg
* fixing some sqs tests
* removing response-parsing tests that are no longer necessary as we're using the botocore response parsing machinery instead of implementing SAX parsing in kombu.
* fixing more SQS tests
* wants a region
* trying to fix py2 parsing of sqs message
* lint
* py2/py2 message header parsing stupidness
* forgot
* python 2 sux
* flake8
* Import boto3 from the right place
* Changes
* Update encode fuction
* Fix lint
* remove some unused things
* removing unused stuff
* ugh
* ugh
* ugh
* landscape ignoring
* shut up, landscape
Tomer Nosrati