* Hook connection pool lifecycle within threadless
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix test
* Fix spell
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Define work lifecycle events for pool
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use isinstance
* Use mocker fixture to pass CI on 3.6 and 3.7
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Tie connection pool into Threadless
* Pass upstream conn pool reference to work instances
* Mark upstream conn pool as optional
* spellcheck
* Fix unused import
* Convert `--local-executor` in an integer flag, defaults to 1 i.e. enabled, use 0 to disable
* Consider any value other than 1 as remote mode
* Use integer to disable local executor mode in integration tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix mypy errors
* Update flags in readme
* Check for type
* Remove pid file check for now
* Update `--local-executor` flag usage
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* do it
* Change defaults
* integrate with ci/cd updates
* Fix ci
* Add dockerfile `SKIP_OPENSSL` option which will allow us to build container without openssl
* Skip openssl for latest tag, add another openssl tag for images with openssl support
* Push separate openssl image to GHCR for every PR
* Add `ProgramNamePlugin`
* Update readme
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove `_compat.py`
* Add suggestions coming from #659
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Optimize `find_http_line` which is in critical path
* Update benchmark results
* Keep the loop hot, TCP no delay, cleanup inactive check periodically
* Check for shutdown signal with tick
* Use non-reentrant `NonBlockingQueue` implementation instead of `queue.Queue`
* Fix listener test
* lint and doc
* Use threads for delegation. Now `run_once` lock before `accept` not `select`
* Add support to use master proxy within proxy pool plugin. When used, proxy pool plugin will be a no-op for the master node
* Fix acceptor tests now that mask is being used
* Use `cached_property` for web server routes
* Use `select(timeout=1)` otherwise acceptor wont join if total blocking
* mypy, flake, doc spell fixes
* R0205: Class `cached_property` inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
* Update `make lib-profile`
* Optimize `utils.find_http_line`
* Pass work to executors within their own multiprocessing lock
* Fix tests
* Add `(_py_class_role, Url)` to fix rtfd :D
* Abstract out a Listener class
* unused
* Use connection instead of manager queue
* For web close connection of client requested via headers
* Remove eventing WIP module
* Sub and Unsub ack
* Fix tests
* mypy and flake8
* comma
* Move callback within EventSubscriber constructor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip test_unix_path_listener on Windows
* Spelling fix
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Allow overriding work_klass via Proxy context manager kwargs
* Decouple acceptor and executor pools
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add `--num_acceptors` flag and better load balancing
* Remove unused
* Lint errors
* Another arg not kwarg
* Move start work staticmethods within ExecutorPool
* mypy fixes
* Update README with `--num-acceptors` flag
* Rename `Proxy.pool` to `Proxy.acceptors`
* Add SetupShutdownContextManager abstraction
* Match --num-acceptors logic with PR description
* Rename executor utility methods and add docstring
* Remove work_klass from constructors and pass it via flags
* Update docstring for pools as they no longer accept a work_klass argument
* Turn work_klass into a flag. main() no longer accepts input_args (only kwargs opts). Similarly, Proxy doesnt accept any input_args now (only kwargs opts)
* Expose default work klass in README
* Expose `HttpProtocolHandler` and `HttpProtocolHandlerPlugin` within `proxy.http` module
* Start to fix tests
* Fix tests
* mypy and flake8
* Trailing comma
* Remove unused var
* Unused arg
* uff
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Explicit `multiprocessing.Manager.shutdown`
Multiprocessing manager is used within eventing core. From doc,
it appears to start a BaseManager which starts a server????
Seriously???? Anyways, using multiprocessing manager is a PITA
and mistake, as it doesn't even give us performance we expect.
Our proxy server can handle more requests than what multiprocess
manager can exchange between processes.
* `--threadless is now ON by default for `Python 3.8+` on `mac` and `linux` environments
* Clarity around why multiprocessing.Manager must be deprecated
* Add `--threaded` flag which can be used to fallback for environments where `--threadless` is now default
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* never used
* Update README
* Use `threaded=True` in tests which were written for threaded model
* Fix issue where sharing manager between global event queue and subscriber can lead to TypeError
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Move Proxy.initialize within FlagParser.initialize. Also move other staticmethods from within proxy class into utils
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused imports
* Fix `import-outside-toplevel` error
* add `make lib-flake8` and `make lib-mypy` targets
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Refactor pool
* mypy fixes
* Fix import (relative)
* Add WebScraper example skeleton & ConnectionPool skeleton
* Add ConnectionPool class
* Integrate ConnectionPool with proxy server (experimental)
* Lint fixes
* Remove unused imports. TODO: Put pool behind a flag. Default to false for now
* Make ConnectionPool multiprocess safe. Later we want to make it safe but without using locks
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused imports
* Return created flag from acquire
* Guard connection pool behind --enable-conn-pool flag
* Flag belongs within connection pool class
* spelling
* self.upstream = None only for pool config
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Build docker from 3.10-alpine
* Bump version to 3.4.0
* Add instructions for how to run dashboard
* Order of menu
* Override dashboard png path until submitted
* Add some doc string for top-level Proxy class. Also some TODOs and warnings regarding PID file overwrite
* Allow HttpProxyBasePlugin implementations to register custom descriptors for read/write events
* Remove hardcoded adblock regex into json config. Update upstream filter to block facebook, not google
* ProxyPoolPlugin and ReverseProxyPlugin must now be updated to use get/read/write descriptor APIs
* Add get/read/write descriptor API for HttpWebServerBasePlugin too
* Surface actual listening port via flags.port
* Go flagless to allow custom user defined flags. Fixes#301
* Add --cache-dir flag for cache plugin (when used with on-disk store)
* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand. This also allows external flags to surface in --help section
* Define --filtered-client-ips flag for FilterByClientIpPlugin
* Stash current changes
* Refactor into connection module
* Response parser state complete when no body expect
* Raise NotImplementedError if invalid state reached within parser
* Initialize MacOS Menubar application
* Dashboard plugin at-least needs a shutdown hook to teardown any thread/processes started by dashboard backend plugin
* Add menu bar icon
* Add respective test directories
* Sync test banners
* Move plugin tests under its own package
* Enable daemon for threads, other this wont shutdown cleanly
* Move tests into individual modules too
* Ensure one test class per file
* Fix docker image after refactoring
* Add github actions workflow for building docker image
* Fix image name
* Setup python required for extracting proxy version
* Version will also require deps
Abhinav Singh