* Add response pkt utility
* Unused import
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tests as some content is now by default gzipped based upon min compression config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused
* Update necessary tests to use `okResponse` utility
* Add option to explicitly disable compression
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Asynchronous `handle_event` and `LocalExecutor` thread
* Bail out on first task completion
* mypy
* Add `helper/benchmark.sh` and fix threaded which must now use asyncio (reduced performance of threaded)
* Print open file diff from `benchmark.sh`
* Add `--local-executor` flag, disabled by default for now until tests are updated
* Async `handle_readables` and `handle_writables` for `HttpProtocolHandlerPlugin` interface (doesnt impact proxy/web plugins for now)
* Async `get_events`
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address tests after async changes
* mypy and flake8
* spelldoc
* `check.py` and trailing comma
* Rename to `_assertions.py`
* Add missing `pytest-mock` and `pytest-asyncio` deps
* Add `pytest-mock` to `pylint` deps
* Correct use of `parameterize` and add `PT007` to flake8 ignores
* Fix mypy hints broken for `< Python3.9`
* Remove usage of `asynccontextmanager` which is not available for all Python versions that `proxy.py` supports
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix for pre-python-3.9 versions
* `AsyncTask` apis `set_name` and `get_name` are not available on all supported versions
* Install setuptools via `lib-dep` until we recommend editable install
* Deprecate support for `Python 3.6`
* Use recommendation suggested here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753840929
* Address recommendation here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753841906
* Make `Threadless` agnostic of `multiprocessing.Process`
* Acceptors must dispatch to local executor in non-blocking fashion
* No daemon for executor processes and fix shutdown logic
* Only return fds from `_selected_events` not all events data
* Refactor logic
* Prefix private methods with `_`
* `work_queue` and not `client_queue`
* Turn `Threadless` into an abstract executor. Introduce `RemoteExecutor`
* Make `LocalExecutor` agnostic of `threading.Thread`
* `LocalExecutor` now implements `Threadless`
* `get_events` and `get_descriptors` now must return int and not sock. `Threadless` now avoids repeated register/unregister and instead make use of `selectors.modify`
* Fix `main` tests
* Apply suggestions from code review
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
* Apply code review recommendations manually
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert back `Any` and use `addr or None`
* Address `flake8`
* Update tests to use `fileno`
* Fix doc build
* Fix doc spell, use tear down and not teardown
* Doc updates
* Add back support for `Python 3.6`
* Acceptors dont need loop initialization
* On Python 3.6 `asyncio.new_event_loop()` is necessary
* Make doc happy
* `--threaded` needs a new event loop for 3.7 too
* Always use `asyncio.new_event_loop()` for threaded mode
Added e2e integration tests (subprocess & curl) for all modes.
* Lint fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
* Remove abstractmethod for proxy plugin base class, remove unused methods from bundled plugins
* Move httpStatusCodes, httpMethods and Url within top-level proxy.http package
* Custom Url parser for our needs
* lint fix
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix proxy_pool plugin as scheme can be None if not present in the Url
* Address the ambiguous ipv6:port scenario along with valid cases
* lint checks
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docstring
* Abstract into `http.parser` module
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix#398 HTTP/1.0 related issue
* lint checks
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Allow overriding work_klass via Proxy context manager kwargs
* Decouple acceptor and executor pools
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add `--num_acceptors` flag and better load balancing
* Remove unused
* Lint errors
* Another arg not kwarg
* Move start work staticmethods within ExecutorPool
* mypy fixes
* Update README with `--num-acceptors` flag
* Rename `Proxy.pool` to `Proxy.acceptors`
* Add SetupShutdownContextManager abstraction
* Match --num-acceptors logic with PR description
* Rename executor utility methods and add docstring
* Remove work_klass from constructors and pass it via flags
* Update docstring for pools as they no longer accept a work_klass argument
* Turn work_klass into a flag. main() no longer accepts input_args (only kwargs opts). Similarly, Proxy doesnt accept any input_args now (only kwargs opts)
* Expose default work klass in README
* Expose `HttpProtocolHandler` and `HttpProtocolHandlerPlugin` within `proxy.http` module
* Start to fix tests
* Fix tests
* mypy and flake8
* Trailing comma
* Remove unused var
* Unused arg
* uff
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Explicit `multiprocessing.Manager.shutdown`
Multiprocessing manager is used within eventing core. From doc,
it appears to start a BaseManager which starts a server????
Seriously???? Anyways, using multiprocessing manager is a PITA
and mistake, as it doesn't even give us performance we expect.
Our proxy server can handle more requests than what multiprocess
manager can exchange between processes.
* `--threadless is now ON by default for `Python 3.8+` on `mac` and `linux` environments
* Clarity around why multiprocessing.Manager must be deprecated
* Add `--threaded` flag which can be used to fallback for environments where `--threadless` is now default
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* never used
* Update README
* Use `threaded=True` in tests which were written for threaded model
* Fix issue where sharing manager between global event queue and subscriber can lead to TypeError
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Move Proxy.initialize within FlagParser.initialize. Also move other staticmethods from within proxy class into utils
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused imports
* Fix `import-outside-toplevel` error
* add `make lib-flake8` and `make lib-mypy` targets
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Rename .server to .upstream
* Lint fixes
* Mark internal methods with _ prefix
* Fix broken test
* lint changes
* Wah, double client :D
* Avoid selector initialization for threadless mode
* remove unused imports
* Now HttpProtocolHandler implements BaseTcpServerHandler
* Consistent return and guard againt upstream.closed
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AcceptorPool as context manager
* Group multiprocessing imports together
* Use com.jaxl bundle identifier as proxy.py will eventually move under jaxl org
* revisit devtools integration :)
* Emit all necessary events for devtools integration
* Lint fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Build docker from 3.10-alpine
* Bump version to 3.4.0
* Add instructions for how to run dashboard
* Order of menu
* Override dashboard png path until submitted
* Add some doc string for top-level Proxy class. Also some TODOs and warnings regarding PID file overwrite
* Allow HttpProxyBasePlugin implementations to register custom descriptors for read/write events
* Remove hardcoded adblock regex into json config. Update upstream filter to block facebook, not google
* ProxyPoolPlugin and ReverseProxyPlugin must now be updated to use get/read/write descriptor APIs
* Add get/read/write descriptor API for HttpWebServerBasePlugin too
* Surface actual listening port via flags.port
* Go flagless to allow custom user defined flags. Fixes#301
* Add --cache-dir flag for cache plugin (when used with on-disk store)
* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand. This also allows external flags to surface in --help section
* Define --filtered-client-ips flag for FilterByClientIpPlugin
* Initial draft of filter_by_url_regex.py
* Add FilterByURLRegexPlugin
* Fix dictionary key & add logging
* Add proper logging
* Add better logging
* Add logging
* move code to handle_client_request
* development logging
* development
* development
* development
* dev
* dev
* dev
* dev
* dev
* dev
* dev
* dev
* dev
* dev
* dev
* Fix blocked log
* Add to FILTER_LIST, some tidy up
* Update FILTER_LIST
* dev
* remove scheme from url
* Add to FILTER_LIST
* Add to FILTER_LIST
* Update FILTER_LIST
* commenting
* Update FILTER_LIST
* After autopep8
* Fix Anomalous backslash in string (pep8)
* Address code quality checks - flake8 F401 & W605
* Address flake8 errors
* Attempt to fix flake8 errors
* Fix linting issues
* Address flake8 W292
* Attempt to create tests
* Add FilterByURLRegexPlugin
* Rename test
* Work on tests
* Work on tests
* Work on tests
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
* Move wrap functionality within respective connection classes. Also decouple websocket client handshake method
* Add a TCP echo client example that works with TCP echo server example
* Use common.pki for interception certificate generation
* Fix tests
* Dont use certificate fields that we dont need, it leads to certificate generation error on Ubuntu
* Prepare for v2.2.0
* npm audit fix
* Add DEFAULT_HTTP_PORT constant
* Use DEFAULT_HTTP_PORT in tests
* Refactor into exception module
* Refactor into inspector module
* Refactor into server module
* Refactor into proxy module
* connection.recv now returns a memoryview
* Make connection.queue also memoryview compliant
* autopep8
* wrap in memoryview as necessary
* Add default timeout for socket_connection and test_embed urllib
* Fix tests
* Skip TestProxyPyEmbedded for now, verifying GitHub actions
* Add timeout for wait_for_server and skip only if GITHUB_ACTIONS env variable is set
* Verify if GitHub Action fails due to wait_for_server spinning forever
* Add test for wait_for_server timeout error exception
* GitHub action hangs irrespective of wait_for_server timeout, disable TestEmbed for GitHub actions
* Initialize MacOS Menubar application
* Dashboard plugin at-least needs a shutdown hook to teardown any thread/processes started by dashboard backend plugin
* Add menu bar icon
* Add respective test directories
* Sync test banners
* Move plugin tests under its own package
* Enable daemon for threads, other this wont shutdown cleanly
Abhinav Singh