Putting a 'rediss'-specific check into a generic connection module feels
like a code smell but there doesn't appear to be another way to intercept
the connection initiation code. It defaults to the least secure form, as
there is no suitable default location for `ca_certs`. The recommendation
would still be to follow the documentation and specify `broker_use_ssl` if
coming from celery.
* Update SimpleQueue for queue_arguments passing
Allow SimpleQueue to pass queue_arguments to Queue object.
This allows SimpleQueue to connect to RabbitMQ queues with custom arguments like 'x-queue-mode'='lazy'.
* Missed indent
* remove trailing space
* add 'queue_args' to arguments (and documentation) for SimpleQueue
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* add test to verify new argument 'queue_args'.
Signed-off-by: Blue Cuenca <blue.cuenca@gmail.com>
* If give queue kwargs broadcast will fail
Because give Broadcast(name='xxx', queue='xxx'),
it will create one specific name for queue,
and if launch more worker for consuming this queue,
this specific name queue already exist then it will not create new queue to
binding fanout exchange.
So all worker will consuming same queue, and exchange only binding this
queue then lose the effect of broadcasting.
* modify test_Broadcast
To fit with the documentation, Queue can now handle a str argument for
the exchange parameter in the constructor, as well as an actual Exchange
object.
Added relevant unit tests to avoid regressions.
Fixes: https://github.com/celery/kombu/issues/903
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
* Regenerate poll()s of Hub and Redis, after shutdown
Allows the same Python process to bring a new Worker
after a shutdown (SystemExit) rescued.
See: https://botbot.me/freenode/celery/2018-04-16/?msg=99046233&page=2
* Hub.poller is now a @property
* Fixed tests that supose .poller to be settable to None
* Test hub.pooler regeneration
* No need to regenerate poller on Redis if the Hub behaves nicely
* Prevent spurious exception on Resource.resize(0)
`Resource.resize()` raises an exception if the pool is in use and the new size is smaller than the old size. However, it also raises this exception when the new size is zero, which should correspond to disabling the pool. Instead of shrinking the pool to zero and releasing all resources, we can simply dequeue all resources and forget about them.
* Add test for removing pool limit when in use
* Fixes#791
* Changing to recommended patch by @georgepsarakis
* Revert "Fixes #791"
This reverts commit 5593505dd9.
* Updated to make tests pass
* Made _ensure_str a private function
* Code formatting for flake8
* Added a mock of the newstr and newbytes classes to create a failing test that simulates the issue with using python-future under 2.7.
* Fix infinite loop in create_loop
fixes https://github.com/celery/celery/issues/3712
Before handling the todo items we "freeze" them by copying them aside and clearing the list.
This way if an item in the todo list appends a new callable to the list itself it will be taken care of in the next iteration of the parent loop instead of producing an infinite loop by adding it to the list we're running on.
* Changed the test to be aligned with the new implementation
* passing flake8
* Avoid copying results with each iteration of the async loop.
* Pop instead of slicing.
* fixed: todos -> todo, fixed test to use MagicMock so we can use the len() method
* MagicMock not supported in 2.7, implemented __len__ on Mock instead
* added entry to changelog
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Re-Adding support for sqlalchemy as it is needed by Apache project Airflow
* Switch Boto2 to Boto3 for SQS messaging
* Fixed region support
* Add SQS FIFO queue support
* Add sensible defaults for message attributes
* Asynchronous support, plus boto3 for region endpoint lookups
* Clean up imports
* Fix Python 2 support
* Fix receive_message tests
* Reformat docstring
* boto3 import changes for CI
* skip tests if boto3 not installed
* skip tests if boto3 not installed
* flake8
* noboto
* ditching boto2. got queue URL fetching, async HTTP request generation and signing working.
* request signing working kinda
* async parsing of SQS message response more or less working
* botocore sqs dep
* ripping out more old boto2 stuff
* removing tests that are no longer valid with boto3/SQS
* fix boto3 dep, min version and no botocore
* no boto2 for test
* cleaning up some SQS tests. fixing header parsing of response to msg
* fixing some sqs tests
* removing response-parsing tests that are no longer necessary as we're using the botocore response parsing machinery instead of implementing SAX parsing in kombu.
* fixing more SQS tests
* wants a region
* trying to fix py2 parsing of sqs message
* lint
* py2/py2 message header parsing stupidness
* forgot
* python 2 sux
* flake8
* Import boto3 from the right place
* Changes
* Update encode fuction
* Fix lint
* remove some unused things
* removing unused stuff
* ugh
* ugh
* ugh
* landscape ignoring
* shut up, landscape
Daniel Blair