Commit Graph

9 Commits

Author SHA1 Message Date
Abhinav Singh 54e74a911f
[isort] Lib modules (#1016)
* isort `proxy.py` main class

* isort init and main

* isort common

* pre-commit fix

* isort dashboard and testing

* isort plugins

* isort core

* Only sort top level http py files

* isort http exception and websocket

* Remove proxy auth plugin from proxy package exports and force discover `PLUGIN_PROXY_AUTH` flags

* isort parser and web server

* no setattr

* isort all

* Enable pre-commit isort hook

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-01-20 15:34:54 +05:30
Sviatoslav Sydorenko fa89944891
Add initial Sphinx docs 2021-11-18 01:36:13 +01:00
Abhinav Singh df7f1c887b
Add `Listener`, Web server close on header, use `Pipe` instead of `Manager` in eventing core (#720)
* Abstract out a Listener class

* unused

* Use connection instead of manager queue

* For web close connection of client requested via headers

* Remove eventing WIP module

* Sub and Unsub ack

* Fix tests

* mypy and flake8

* comma

* Move callback within EventSubscriber constructor

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip test_unix_path_listener on Windows

* Spelling fix

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-10 20:47:28 +05:30
Abhinav Singh 0a038ce8be
Add `--num-acceptors` flag + Allow `work_klass` via `Proxy` context manager kwargs (#714)
* Allow overriding work_klass via Proxy context manager kwargs

* Decouple acceptor and executor pools

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add `--num_acceptors` flag and better load balancing

* Remove unused

* Lint errors

* Another arg not kwarg

* Move start work staticmethods within ExecutorPool

* mypy fixes

* Update README with `--num-acceptors` flag

* Rename `Proxy.pool` to `Proxy.acceptors`

* Add SetupShutdownContextManager abstraction

* Match --num-acceptors logic with PR description

* Rename executor utility methods and add docstring

* Remove work_klass from constructors and pass it via flags

* Update docstring for pools as they no longer accept a work_klass argument

* Turn work_klass into a flag.  main() no longer accepts input_args (only kwargs opts).  Similarly, Proxy doesnt accept any input_args now (only kwargs opts)

* Expose default work klass in README

* Expose `HttpProtocolHandler` and `HttpProtocolHandlerPlugin` within `proxy.http` module

* Start to fix tests

* Fix tests

* mypy and flake8

* Trailing comma

* Remove unused var

* Unused arg

* uff

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-10 04:27:40 +05:30
Abhinav Singh 98e6d0b3d4
`--threadless` default for `Python 3.8+` on `mac` and `linux` (#710)
* Explicit `multiprocessing.Manager.shutdown`

Multiprocessing manager is used within eventing core. From doc,
it appears to start a BaseManager which starts a server????
Seriously???? Anyways, using multiprocessing manager is a PITA
and mistake, as it doesn't even give us performance we expect.
Our proxy server can handle more requests than what multiprocess
manager can exchange between processes.

* `--threadless is now ON by default for `Python 3.8+` on `mac` and `linux` environments

* Clarity around why multiprocessing.Manager must be deprecated

* Add `--threaded` flag which can be used to fallback for environments where `--threadless` is now default

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* never used

* Update README

* Use `threaded=True` in tests which were written for threaded model

* Fix issue where sharing manager between global event queue and subscriber can lead to TypeError

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-09 01:34:17 +05:30
Abhinav Singh aadcc10813
`EventManager` is also a context manager (#709)
* `EventManager` is also a context manager

* unused

* Rename `EventManager.event_queue` to `EventManager.queue`
2021-11-08 23:02:36 +05:30
Abhinav Singh a8e3966525
Put core flags where they belong (#702)
* Move flags to where they belong

* Move `get_default_plugins` within FlagParser as it depends upon args

TODO: We need plugin dependency system

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-08 03:21:42 +05:30
Sviatoslav Sydorenko 05a78567ac
Enable the `add-trailing-comma` pre-commit fixer (#661)
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
2021-11-04 16:58:36 +05:30
Abhinav Singh d4ee4fa15c
Async proxy pool, Event manager, Custom access log, Expose loop to plugins (#645)
* Async proxy pool

* Async proxy pool

* Late upstream initialization and exception guards

* Close upstream proxy connection on client connection close

* Refactor into EventManager

* Fix tests accounting in the event manager

* Ensure each process initializes logger

* pragma no cover

* Teardown connection when proxy pool upstream proxy closes

* Add ability to customize access log format and add additional context to it

* Maintain total size for response bytes in access logs

* Fix tests broken due to new plugin methods missing mock

* Update pubsub_eventing to use EventManager to avoid entire bootstrapping step
2021-11-01 01:19:19 +05:30