Missing redis key containg set of queues bound to queue is caused
by removal all queues from the exchange. Hence, we should not raise an
exception but just return empty list of queues instead. This commit
fixes e.g. case publishing message against empty exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
exchange.declare(channel=conn.default_channel)
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
But it also fixes the case when last queue is unbound from exchange and
after publishing to this exchange:
import kombu
conn = kombu.Connection('redis://')
exchange = kombu.Exchange('name', type='direct')
queue1 = kombu.Queue('queue1', exchange=exchange, routing_key='queue1')
exchange.declare(channel=conn.default_channel)
queue1 = queue1.bind(channel=conn.default_channel)
queue1.declare()
producer = conn.Producer()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
queue1.delete()
producer.publish(
{'hello': 'world'},
exchange=exchange,
routing_key='queue1'
)
* enable pre-commit
* use extend-ignore for flake8
* manual flake8 fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update kombu/__init__.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add transports based on Azure PaaS
This pull request adds two new transport implementations:
- `azurestoragequeues` is implemented on top of Azure Storage
Queues [1]. This offers a simple but scalable and low-cost PaaS
transport for Celery users in Azure. The transport is intended to be
used in conjunction with the Azure Block Blob Storage backend [2].
- `azureservicebus` is implemented on top of Azure Service Bus [3] and
offers PaaS support for more demanding Celery workloads in Azure. The
transport is intended to be used in conjunction with the Azure
CosmosDB backend [4].
This pull request was created together with @ankurokok, @dkisselev,
@evandropaula, @martinpeck and @michaelperel.
[1] https://azure.microsoft.com/en-us/services/storage/queues/
[2] https://github.com/celery/celery/pull/4685
[3] https://azure.microsoft.com/en-us/services/service-bus/
[4] https://github.com/celery/celery/pull/4720
* Exclude Azure transports from code coverage
There is test coverage for the transports but the tests require Azure
credentials to run (passed via environment variables) so codecov doesn't
exercise them.
* Remove env vars to configure transport
* Remove abbreviations
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Switch Boto2 to Boto3 for SQS messaging
* Fixed region support
* Add SQS FIFO queue support
* Add sensible defaults for message attributes
* Asynchronous support, plus boto3 for region endpoint lookups
* Clean up imports
* Fix Python 2 support
* Fix receive_message tests
* Reformat docstring
* boto3 import changes for CI
* skip tests if boto3 not installed
* skip tests if boto3 not installed
* flake8
* noboto
* ditching boto2. got queue URL fetching, async HTTP request generation and signing working.
* request signing working kinda
* async parsing of SQS message response more or less working
* botocore sqs dep
* ripping out more old boto2 stuff
* removing tests that are no longer valid with boto3/SQS
* fix boto3 dep, min version and no botocore
* no boto2 for test
* cleaning up some SQS tests. fixing header parsing of response to msg
* fixing some sqs tests
* removing response-parsing tests that are no longer necessary as we're using the botocore response parsing machinery instead of implementing SAX parsing in kombu.
* fixing more SQS tests
* wants a region
* trying to fix py2 parsing of sqs message
* lint
* py2/py2 message header parsing stupidness
* forgot
* python 2 sux
* flake8
* Import boto3 from the right place
* Changes
* Update encode fuction
* Fix lint
* remove some unused things
* removing unused stuff
* ugh
* ugh
* ugh
* landscape ignoring
* shut up, landscape
Matus Valo