The changed `zadd()` behaviour in redis-py 3 states:
https://github.com/andymccurdy/redis-py#mset-msetnx-and-zadd
> For ZADD, the dict is a mapping of element-names -> score.
So the dictionary generated for `zadd()` should be flipped around to have the
`delivery_tag` as the key and the score as value. This commit fixes that.
* Add boto3 default region before hardcoded default
* Add unit tests for SQS region default behavior
* Import boto3 only inside the test
The library may not be available
Further to #946 this fixes the underlying issue in a easy-to-upgrade way
for end users, many of whom will have Redis installed via other means.
By having this check here and supporting both versions concurrently it
makes it easier for end users, and to use celery/kombu in projects that
use Redis elsewhere.
With this change it is possibly worth reverting #946
Redis-py 3.0.0 was released today and introduced a backwards incompatible changes.
9b03af26dc/CHANGES (L27-L29)
This isn't the "best" fix but is the quickest
* On restore changes message visibility instead of send new message
* Acknowledge message on hub close
* Use sqs instead of async sqs to delete message
* changes itertools to range
* Empty Hub ready
* fixed test_poller_regeneration_on_access
* Fixed typo in comment
* Simplify loop to process self._ready
* Added test case for redelivered _put
* Lint fixes
* Added test case for delete_message call
* Fix a deprecation warning about logger.warn()
This fixes a deprecation warning from the standard library's logging module:
> The 'warn' method is deprecated, use 'warning' instead
* Update test
* Add transports based on Azure PaaS
This pull request adds two new transport implementations:
- `azurestoragequeues` is implemented on top of Azure Storage
Queues [1]. This offers a simple but scalable and low-cost PaaS
transport for Celery users in Azure. The transport is intended to be
used in conjunction with the Azure Block Blob Storage backend [2].
- `azureservicebus` is implemented on top of Azure Service Bus [3] and
offers PaaS support for more demanding Celery workloads in Azure. The
transport is intended to be used in conjunction with the Azure
CosmosDB backend [4].
This pull request was created together with @ankurokok, @dkisselev,
@evandropaula, @martinpeck and @michaelperel.
[1] https://azure.microsoft.com/en-us/services/storage/queues/
[2] https://github.com/celery/celery/pull/4685
[3] https://azure.microsoft.com/en-us/services/service-bus/
[4] https://github.com/celery/celery/pull/4720
* Exclude Azure transports from code coverage
There is test coverage for the transports but the tests require Azure
credentials to run (passed via environment variables) so codecov doesn't
exercise them.
* Remove env vars to configure transport
* Remove abbreviations
Putting a 'rediss'-specific check into a generic connection module feels
like a code smell but there doesn't appear to be another way to intercept
the connection initiation code. It defaults to the least secure form, as
there is no suitable default location for `ca_certs`. The recommendation
would still be to follow the documentation and specify `broker_use_ssl` if
coming from celery.
* Update SimpleQueue for queue_arguments passing
Allow SimpleQueue to pass queue_arguments to Queue object.
This allows SimpleQueue to connect to RabbitMQ queues with custom arguments like 'x-queue-mode'='lazy'.
* Missed indent
* remove trailing space
* add 'queue_args' to arguments (and documentation) for SimpleQueue
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* add test to verify new argument 'queue_args'.
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* If give queue kwargs broadcast will fail
Because give Broadcast(name='xxx', queue='xxx'),
it will create one specific name for queue,
and if launch more worker for consuming this queue,
this specific name queue already exist then it will not create new queue to
binding fanout exchange.
So all worker will consuming same queue, and exchange only binding this
queue then lose the effect of broadcasting.
* modify test_Broadcast
To fit with the documentation, Queue can now handle a str argument for
the exchange parameter in the constructor, as well as an actual Exchange
object.
Added relevant unit tests to avoid regressions.
Fixes: https://github.com/celery/kombu/issues/903
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
* Regenerate poll()s of Hub and Redis, after shutdown
Allows the same Python process to bring a new Worker
after a shutdown (SystemExit) rescued.
See: https://botbot.me/freenode/celery/2018-04-16/?msg=99046233&page=2
* Hub.poller is now a @property
* Fixed tests that supose .poller to be settable to None
* Test hub.pooler regeneration
* No need to regenerate poller on Redis if the Hub behaves nicely
I've gone through the commits since the last tag and added them, grouping the minor tweaks/fixups and doc changes. I think I've got everything.
Incidentally I've left a question in #689 but that needn't prevent release.
Alexey Popravka