* Fix a deprecation warning about logger.warn()
This fixes a deprecation warning from the standard library's logging module:
> The 'warn' method is deprecated, use 'warning' instead
* Update test
* Add transports based on Azure PaaS
This pull request adds two new transport implementations:
- `azurestoragequeues` is implemented on top of Azure Storage
Queues [1]. This offers a simple but scalable and low-cost PaaS
transport for Celery users in Azure. The transport is intended to be
used in conjunction with the Azure Block Blob Storage backend [2].
- `azureservicebus` is implemented on top of Azure Service Bus [3] and
offers PaaS support for more demanding Celery workloads in Azure. The
transport is intended to be used in conjunction with the Azure
CosmosDB backend [4].
This pull request was created together with @ankurokok, @dkisselev,
@evandropaula, @martinpeck and @michaelperel.
[1] https://azure.microsoft.com/en-us/services/storage/queues/
[2] https://github.com/celery/celery/pull/4685
[3] https://azure.microsoft.com/en-us/services/service-bus/
[4] https://github.com/celery/celery/pull/4720
* Exclude Azure transports from code coverage
There is test coverage for the transports but the tests require Azure
credentials to run (passed via environment variables) so codecov doesn't
exercise them.
* Remove env vars to configure transport
* Remove abbreviations
Putting a 'rediss'-specific check into a generic connection module feels
like a code smell but there doesn't appear to be another way to intercept
the connection initiation code. It defaults to the least secure form, as
there is no suitable default location for `ca_certs`. The recommendation
would still be to follow the documentation and specify `broker_use_ssl` if
coming from celery.
* Update SimpleQueue for queue_arguments passing
Allow SimpleQueue to pass queue_arguments to Queue object.
This allows SimpleQueue to connect to RabbitMQ queues with custom arguments like 'x-queue-mode'='lazy'.
* Missed indent
* remove trailing space
* add 'queue_args' to arguments (and documentation) for SimpleQueue
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* add test to verify new argument 'queue_args'.
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* If give queue kwargs broadcast will fail
Because give Broadcast(name='xxx', queue='xxx'),
it will create one specific name for queue,
and if launch more worker for consuming this queue,
this specific name queue already exist then it will not create new queue to
binding fanout exchange.
So all worker will consuming same queue, and exchange only binding this
queue then lose the effect of broadcasting.
* modify test_Broadcast
To fit with the documentation, Queue can now handle a str argument for
the exchange parameter in the constructor, as well as an actual Exchange
object.
Added relevant unit tests to avoid regressions.
Fixes: https://github.com/celery/kombu/issues/903
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
* Regenerate poll()s of Hub and Redis, after shutdown
Allows the same Python process to bring a new Worker
after a shutdown (SystemExit) rescued.
See: https://botbot.me/freenode/celery/2018-04-16/?msg=99046233&page=2
* Hub.poller is now a @property
* Fixed tests that supose .poller to be settable to None
* Test hub.pooler regeneration
* No need to regenerate poller on Redis if the Hub behaves nicely
I've gone through the commits since the last tag and added them, grouping the minor tweaks/fixups and doc changes. I think I've got everything.
Incidentally I've left a question in #689 but that needn't prevent release.
* Prevent spurious exception on Resource.resize(0)
`Resource.resize()` raises an exception if the pool is in use and the new size is smaller than the old size. However, it also raises this exception when the new size is zero, which should correspond to disabling the pool. Instead of shrinking the pool to zero and releasing all resources, we can simply dequeue all resources and forget about them.
* Add test for removing pool limit when in use
* Fixes#791
* Changing to recommended patch by @georgepsarakis
* Revert "Fixes #791"
This reverts commit 5593505dd9.
* Updated to make tests pass
* Made _ensure_str a private function
* Code formatting for flake8
* Added a mock of the newstr and newbytes classes to create a failing test that simulates the issue with using python-future under 2.7.
Vsevolod Strukchinsky