* Added pattern/matcher to Mailbox
* pattern/match for kombu 4
* Ensure kombu.matcher is covered by our documentation.
* Adds test_matcher & pidbox unit tests.
* Added tests to ensure exception is raised when matcher is not registered.
* Adds to test for destination passed in to process.
* fixed rediss url parsing
* added ssl parameter test
* added ssl parameter test
* added ssl parameter test
* added ssl parameter test
* added ssl parameter test
* added no ssl module parse_url test
* added no ssl module parse_url test
* added no ssl module parse_url test
* convert to list comprehension
* simulate no ssl in test
* add missing logger
The problem is that `self.connection.client.alt` is only populated when
there's more than one client URL provided, e.g.
`"sentinel://foo;sentinel://bar"`. It will also always contain all URLs,
including the primary/first entry.
So if the `alt` list is empty, it (usually) means there was only one
client URL provided.
I also took the liberty to perform name and type changes to be more in
line with the examples and documentation in the `redis` library. The
argument is `sentinels`, not `connection_list`. And it's of the type
`List[Tuple[str, int]]`, not `List[List[str, str]]`.
Fixes#1004
* Moving the handling of Sentinel failures to the redis library itself.
Currently Celery dies if the first Sentinel servive on the list is down,
as in if you lose a machine.
* Removing the use of unittest.
* Changing how an empty list is defined.
* Make the pyro transport available and fix it for recent Pyro4 version.
Implemented the missing Pyro Kombu broker to actually be able to use the Pyro transport
* code style
* Add boto3 default region before hardcoded default
* Add unit tests for SQS region default behavior
* Import boto3 only inside the test
The library may not be available
Further to #946 this fixes the underlying issue in a easy-to-upgrade way
for end users, many of whom will have Redis installed via other means.
By having this check here and supporting both versions concurrently it
makes it easier for end users, and to use celery/kombu in projects that
use Redis elsewhere.
With this change it is possibly worth reverting #946
* On restore changes message visibility instead of send new message
* Acknowledge message on hub close
* Use sqs instead of async sqs to delete message
* changes itertools to range
* Empty Hub ready
* fixed test_poller_regeneration_on_access
* Fixed typo in comment
* Simplify loop to process self._ready
* Added test case for redelivered _put
* Lint fixes
* Added test case for delete_message call
* Fix a deprecation warning about logger.warn()
This fixes a deprecation warning from the standard library's logging module:
> The 'warn' method is deprecated, use 'warning' instead
* Update test
* Add transports based on Azure PaaS
This pull request adds two new transport implementations:
- `azurestoragequeues` is implemented on top of Azure Storage
Queues [1]. This offers a simple but scalable and low-cost PaaS
transport for Celery users in Azure. The transport is intended to be
used in conjunction with the Azure Block Blob Storage backend [2].
- `azureservicebus` is implemented on top of Azure Service Bus [3] and
offers PaaS support for more demanding Celery workloads in Azure. The
transport is intended to be used in conjunction with the Azure
CosmosDB backend [4].
This pull request was created together with @ankurokok, @dkisselev,
@evandropaula, @martinpeck and @michaelperel.
[1] https://azure.microsoft.com/en-us/services/storage/queues/
[2] https://github.com/celery/celery/pull/4685
[3] https://azure.microsoft.com/en-us/services/service-bus/
[4] https://github.com/celery/celery/pull/4720
* Exclude Azure transports from code coverage
There is test coverage for the transports but the tests require Azure
credentials to run (passed via environment variables) so codecov doesn't
exercise them.
* Remove env vars to configure transport
* Remove abbreviations
Putting a 'rediss'-specific check into a generic connection module feels
like a code smell but there doesn't appear to be another way to intercept
the connection initiation code. It defaults to the least secure form, as
there is no suitable default location for `ca_certs`. The recommendation
would still be to follow the documentation and specify `broker_use_ssl` if
coming from celery.
* Update SimpleQueue for queue_arguments passing
Allow SimpleQueue to pass queue_arguments to Queue object.
This allows SimpleQueue to connect to RabbitMQ queues with custom arguments like 'x-queue-mode'='lazy'.
* Missed indent
* remove trailing space
* add 'queue_args' to arguments (and documentation) for SimpleQueue
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* add test to verify new argument 'queue_args'.
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* If give queue kwargs broadcast will fail
Because give Broadcast(name='xxx', queue='xxx'),
it will create one specific name for queue,
and if launch more worker for consuming this queue,
this specific name queue already exist then it will not create new queue to
binding fanout exchange.
So all worker will consuming same queue, and exchange only binding this
queue then lose the effect of broadcasting.
* modify test_Broadcast
To fit with the documentation, Queue can now handle a str argument for
the exchange parameter in the constructor, as well as an actual Exchange
object.
Added relevant unit tests to avoid regressions.
Fixes: https://github.com/celery/kombu/issues/903
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
* Regenerate poll()s of Hub and Redis, after shutdown
Allows the same Python process to bring a new Worker
after a shutdown (SystemExit) rescued.
See: https://botbot.me/freenode/celery/2018-04-16/?msg=99046233&page=2
* Hub.poller is now a @property
* Fixed tests that supose .poller to be settable to None
* Test hub.pooler regeneration
* No need to regenerate poller on Redis if the Hub behaves nicely
* Prevent spurious exception on Resource.resize(0)
`Resource.resize()` raises an exception if the pool is in use and the new size is smaller than the old size. However, it also raises this exception when the new size is zero, which should correspond to disabling the pool. Instead of shrinking the pool to zero and releasing all resources, we can simply dequeue all resources and forget about them.
* Add test for removing pool limit when in use
* Fixes#791
* Changing to recommended patch by @georgepsarakis
* Revert "Fixes #791"
This reverts commit 5593505dd9.
* Updated to make tests pass
* Made _ensure_str a private function
* Code formatting for flake8
* Added a mock of the newstr and newbytes classes to create a failing test that simulates the issue with using python-future under 2.7.
* Fix infinite loop in create_loop
fixes https://github.com/celery/celery/issues/3712
Before handling the todo items we "freeze" them by copying them aside and clearing the list.
This way if an item in the todo list appends a new callable to the list itself it will be taken care of in the next iteration of the parent loop instead of producing an infinite loop by adding it to the list we're running on.
* Changed the test to be aligned with the new implementation
* passing flake8
* Avoid copying results with each iteration of the async loop.
* Pop instead of slicing.
* fixed: todos -> todo, fixed test to use MagicMock so we can use the len() method
* MagicMock not supported in 2.7, implemented __len__ on Mock instead
* added entry to changelog
Jason Held