Commit Graph

8 Commits

Author SHA1 Message Date
Abhinav Singh f921f56803
`Work` can also be `TcpServerConnection`, not just `TcpClientConnection` (#906)
* Work can also be TcpServerConnection, not just TcpClientConnection.  More over, it can be any generic work type

* Add py_class_role and py_obj

* Port internal integration tests into public repo

* Fix proxy py addr

* Use cross-platform compat shasum
2021-12-26 20:51:31 +05:30
Abhinav Singh 13c64b5357
Introduce `ProgramNamePlugin` plugin (#904)
* Add `ProgramNamePlugin`

* Update readme

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove `_compat.py`

* Add suggestions coming from #659

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-12-25 12:40:25 +05:30
Abhinav Singh 44d72431e3
Async `get_events`, `handle_event`, `handle_readables`, `handle_writables` (#769)
* Asynchronous `handle_event` and `LocalExecutor` thread

* Bail out on first task completion

* mypy

* Add `helper/benchmark.sh` and fix threaded which must now use asyncio (reduced performance of threaded)

* Print open file diff from `benchmark.sh`

* Add `--local-executor` flag, disabled by default for now until tests are updated

* Async `handle_readables` and `handle_writables` for `HttpProtocolHandlerPlugin` interface (doesnt impact proxy/web plugins for now)

* Async `get_events`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address tests after async changes

* mypy and flake8

* spelldoc

* `check.py` and trailing comma

* Rename to `_assertions.py`

* Add missing `pytest-mock` and `pytest-asyncio` deps

* Add `pytest-mock` to `pylint` deps

* Correct use of `parameterize` and add `PT007` to flake8 ignores

* Fix mypy hints broken for `< Python3.9`

* Remove usage of `asynccontextmanager` which is not available for all Python versions that `proxy.py` supports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix for pre-python-3.9 versions

* `AsyncTask` apis `set_name` and `get_name` are not available on all supported versions

* Install setuptools via `lib-dep` until we recommend editable install

* Deprecate support for `Python 3.6`

* Use recommendation suggested here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753840929

* Address recommendation here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753841906

* Make `Threadless` agnostic of `multiprocessing.Process`

* Acceptors must dispatch to local executor in non-blocking fashion

* No daemon for executor processes and fix shutdown logic

* Only return fds from `_selected_events` not all events data

* Refactor logic

* Prefix private methods with `_`

* `work_queue` and not `client_queue`

* Turn `Threadless` into an abstract executor. Introduce `RemoteExecutor`

* Make `LocalExecutor` agnostic of `threading.Thread`

* `LocalExecutor` now implements `Threadless`

* `get_events` and `get_descriptors` now must return int and not sock.  `Threadless` now avoids repeated register/unregister and instead make use of `selectors.modify`

* Fix `main` tests

* Apply suggestions from code review

Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>

* Apply code review recommendations manually

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert back `Any` and use `addr or None`

* Address `flake8`

* Update tests to use `fileno`

* Fix doc build

* Fix doc spell, use tear down and not teardown

* Doc updates

* Add back support for `Python 3.6`

* Acceptors dont need loop initialization

* On Python 3.6 `asyncio.new_event_loop()` is necessary

* Make doc happy

* `--threaded` needs a new event loop for 3.7 too

* Always use `asyncio.new_event_loop()` for threaded mode

Added e2e integration tests (subprocess & curl) for all modes.

* Lint fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
2021-11-23 15:02:00 +05:30
Abhinav Singh 8fdddfd199
Acceptors performance (#767)
* Use threads for delegation. Now `run_once` lock before `accept` not `select`

* Add support to use master proxy within proxy pool plugin.  When used, proxy pool plugin will be a no-op for the master node

* Fix acceptor tests now that mask is being used

* Use `cached_property` for web server routes

* Use `select(timeout=1)` otherwise acceptor wont join if total blocking

* mypy, flake, doc spell fixes

* R0205: Class `cached_property` inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
2021-11-20 19:42:38 +05:30
Abhinav Singh 42e35188f4
[IntegrationTest] Use `127.0.0.1` as target address and a random port (#756)
* Use `127.0.0.1` as target address and a random port

* Fix spellcheck-docs
2021-11-19 16:49:18 +05:30
Sviatoslav Sydorenko 22a2705e1d
Move GHA e2e integration job to pytest (#746)
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>
2021-11-17 20:01:16 +05:30
Sviatoslav Sydorenko 1bd97678c8
Correct spelling mistakes caught by `codespell` (#656)
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
2021-11-03 17:33:05 +05:30
Abhinav Singh e84c212465
Integration testing (#243)
* Add tests for public/private/csr generation

* Add integration testing skeleton for mac and ubuntu

* Merge integration within lib test to avoid too many workflows

* Disable integration testing on windows for now

* Use sudo to start integration test script as lsof fails on MacOS.

lsof: WARNING: can't stat() vmhgfs file system

* Add basic integration testing for now to assert proxy works as expected when started out of develop branch

* Add a call to inbuilt http server to verify it works

* wait for server to accept requests
2019-12-25 16:39:18 -08:00