Commit Graph

9 Commits

Author SHA1 Message Date
Abhinav Singh 90820b01f5
[ProxyPoolPlugin] Avoid remote proxy of private IP requests (#833)
* Avoid proxy of requests to private IP within `ProxyPoolPlugin`

* Fix tests

* spell fix
2021-12-01 05:29:52 +05:30
Abhinav Singh 44d72431e3
Async `get_events`, `handle_event`, `handle_readables`, `handle_writables` (#769)
* Asynchronous `handle_event` and `LocalExecutor` thread

* Bail out on first task completion

* mypy

* Add `helper/benchmark.sh` and fix threaded which must now use asyncio (reduced performance of threaded)

* Print open file diff from `benchmark.sh`

* Add `--local-executor` flag, disabled by default for now until tests are updated

* Async `handle_readables` and `handle_writables` for `HttpProtocolHandlerPlugin` interface (doesnt impact proxy/web plugins for now)

* Async `get_events`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address tests after async changes

* mypy and flake8

* spelldoc

* `check.py` and trailing comma

* Rename to `_assertions.py`

* Add missing `pytest-mock` and `pytest-asyncio` deps

* Add `pytest-mock` to `pylint` deps

* Correct use of `parameterize` and add `PT007` to flake8 ignores

* Fix mypy hints broken for `< Python3.9`

* Remove usage of `asynccontextmanager` which is not available for all Python versions that `proxy.py` supports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix for pre-python-3.9 versions

* `AsyncTask` apis `set_name` and `get_name` are not available on all supported versions

* Install setuptools via `lib-dep` until we recommend editable install

* Deprecate support for `Python 3.6`

* Use recommendation suggested here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753840929

* Address recommendation here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753841906

* Make `Threadless` agnostic of `multiprocessing.Process`

* Acceptors must dispatch to local executor in non-blocking fashion

* No daemon for executor processes and fix shutdown logic

* Only return fds from `_selected_events` not all events data

* Refactor logic

* Prefix private methods with `_`

* `work_queue` and not `client_queue`

* Turn `Threadless` into an abstract executor. Introduce `RemoteExecutor`

* Make `LocalExecutor` agnostic of `threading.Thread`

* `LocalExecutor` now implements `Threadless`

* `get_events` and `get_descriptors` now must return int and not sock.  `Threadless` now avoids repeated register/unregister and instead make use of `selectors.modify`

* Fix `main` tests

* Apply suggestions from code review

Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>

* Apply code review recommendations manually

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert back `Any` and use `addr or None`

* Address `flake8`

* Update tests to use `fileno`

* Fix doc build

* Fix doc spell, use tear down and not teardown

* Doc updates

* Add back support for `Python 3.6`

* Acceptors dont need loop initialization

* On Python 3.6 `asyncio.new_event_loop()` is necessary

* Make doc happy

* `--threaded` needs a new event loop for 3.7 too

* Always use `asyncio.new_event_loop()` for threaded mode

Added e2e integration tests (subprocess & curl) for all modes.

* Lint fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
2021-11-23 15:02:00 +05:30
Sviatoslav Sydorenko fa89944891
Add initial Sphinx docs 2021-11-18 01:36:13 +01:00
Abhinav Singh 7f1470e3a9
No abstract method for proxy plugin (#738)
* Remove abstractmethod for proxy plugin base class, remove unused methods from bundled plugins

* Move httpStatusCodes, httpMethods and Url within top-level proxy.http package
2021-11-15 03:17:12 +05:30
Sviatoslav Sydorenko 05a78567ac
Enable the `add-trailing-comma` pre-commit fixer (#661)
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
2021-11-04 16:58:36 +05:30
Abhinav Singh 7448c44cc6
v3.4.0 (#638)
* Build docker from 3.10-alpine

* Bump version to 3.4.0

* Add instructions for how to run dashboard

* Order of menu

* Override dashboard png path until submitted

* Add some doc string for top-level Proxy class.  Also some TODOs and warnings regarding PID file overwrite

* Allow HttpProxyBasePlugin implementations to register custom descriptors for read/write events

* Remove hardcoded adblock regex into json config. Update upstream filter to block facebook, not google

* ProxyPoolPlugin and ReverseProxyPlugin must now be updated to use get/read/write descriptor APIs

* Add get/read/write descriptor API for HttpWebServerBasePlugin too

* Surface actual listening port via flags.port
2021-10-30 04:32:05 +05:30
Abhinav Singh 0744cd8e7f
Go flagless to allow custom user defined flags. (#452)
* Go flagless to allow custom user defined flags. Fixes #301

* Add --cache-dir flag for cache plugin (when used with on-disk store)

* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand.  This also allows external flags to surface in --help section

* Define --filtered-client-ips flag for FilterByClientIpPlugin
2020-10-14 10:51:56 +05:30
Abhinav Singh c884338f42
Core acceptor pool doc, cleanup and standalone example (#393)
* Better document acceptor module and add a TCP Echo Server example

* autopep8 formating

* Rename ThreadlessWork --> Work class

* Make initialize, is_inactive and shutdown as optional interface methods.

Also introduce Readables & Writables custom types.

* Move websocket code into its own module

* Add websocket client example

* Cleanup websocket client
2020-07-07 18:01:49 +05:30
Abhinav Singh 1b0ed923d7
Refactor plugin base classes for plugin specific flags (#388)
* Update to latest code signing recommendations

* Move HttpProtocolHandlerPlugin into separate file

* Dont add subject attributes if not provided by upstream. Also handle subprocess.TimeoutExpired raised during certificate generation.  Instead of retries, we simply close the connection on timeout

* Remove plugin specific flag initialization methods for now
2020-07-04 18:17:11 +05:30