Commit Graph

17 Commits

Author SHA1 Message Date
Abhinav Singh dd2476f02a
Refactor into separate `Work` module (#977)
* work module

* Fix imports

* String based typing for multiprocessing.synchronize

* Fix `test_accepts_client_from_server_socket`

* Move staticmethod outside of threadless pool class

* Fix doc build

* Fix test mock

* mp grouped together

* pylint happy

* import only for type checking

* doc build

* wrong import order
2022-01-13 16:07:51 +05:30
Abhinav Singh 35e9c57af7
`isort` everything except lib (for now) (#952)
* isort the tests folder

* Carry over changes from #672

* Disable pre-commit

* Revert flake8 config change

* isort examples too
2022-01-10 13:28:22 +05:30
Abhinav Singh f6214a46c9
Move `UpstreamConnectionPool` lifecycle within `Threadless` (#917)
* Tie connection pool into Threadless

* Pass upstream conn pool reference to work instances

* Mark upstream conn pool as optional

* spellcheck

* Fix unused import
2021-12-28 13:51:20 +05:30
Abhinav Singh 99856a6aa7
Change `--local-executor` flag semantics (#907)
* Convert `--local-executor` in an integer flag, defaults to 1 i.e. enabled, use 0 to disable

* Consider any value other than 1 as remote mode

* Use integer to disable local executor mode in integration tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix mypy errors

* Update flags in readme

* Check for type

* Remove pid file check for now

* Update `--local-executor` flag usage

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-12-27 10:23:38 +05:30
Abhinav Singh 904617eca4
Update defaults for `--hostname` and `--local-executor` (#905)
* do it

* Change defaults

* integrate with ci/cd updates

* Fix ci

* Add dockerfile `SKIP_OPENSSL` option which will allow us to build container without openssl

* Skip openssl for latest tag, add another openssl tag for images with openssl support

* Push separate openssl image to GHCR for every PR
2021-12-25 17:41:15 +05:30
Abhinav Singh 8fdddfd199
Acceptors performance (#767)
* Use threads for delegation. Now `run_once` lock before `accept` not `select`

* Add support to use master proxy within proxy pool plugin.  When used, proxy pool plugin will be a no-op for the master node

* Fix acceptor tests now that mask is being used

* Use `cached_property` for web server routes

* Use `select(timeout=1)` otherwise acceptor wont join if total blocking

* mypy, flake, doc spell fixes

* R0205: Class `cached_property` inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
2021-11-20 19:42:38 +05:30
Abhinav Singh c06cb75b79
Enhancements (#763)
* Update `make lib-profile`

* Optimize `utils.find_http_line`

* Pass work to executors within their own multiprocessing lock

* Fix tests

* Add `(_py_class_role, Url)` to fix rtfd :D
2021-11-20 03:04:15 +05:30
Abhinav Singh df7f1c887b
Add `Listener`, Web server close on header, use `Pipe` instead of `Manager` in eventing core (#720)
* Abstract out a Listener class

* unused

* Use connection instead of manager queue

* For web close connection of client requested via headers

* Remove eventing WIP module

* Sub and Unsub ack

* Fix tests

* mypy and flake8

* comma

* Move callback within EventSubscriber constructor

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip test_unix_path_listener on Windows

* Spelling fix

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-10 20:47:28 +05:30
Abhinav Singh 0a038ce8be
Add `--num-acceptors` flag + Allow `work_klass` via `Proxy` context manager kwargs (#714)
* Allow overriding work_klass via Proxy context manager kwargs

* Decouple acceptor and executor pools

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add `--num_acceptors` flag and better load balancing

* Remove unused

* Lint errors

* Another arg not kwarg

* Move start work staticmethods within ExecutorPool

* mypy fixes

* Update README with `--num-acceptors` flag

* Rename `Proxy.pool` to `Proxy.acceptors`

* Add SetupShutdownContextManager abstraction

* Match --num-acceptors logic with PR description

* Rename executor utility methods and add docstring

* Remove work_klass from constructors and pass it via flags

* Update docstring for pools as they no longer accept a work_klass argument

* Turn work_klass into a flag.  main() no longer accepts input_args (only kwargs opts).  Similarly, Proxy doesnt accept any input_args now (only kwargs opts)

* Expose default work klass in README

* Expose `HttpProtocolHandler` and `HttpProtocolHandlerPlugin` within `proxy.http` module

* Start to fix tests

* Fix tests

* mypy and flake8

* Trailing comma

* Remove unused var

* Unused arg

* uff

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-10 04:27:40 +05:30
Abhinav Singh 98e6d0b3d4
`--threadless` default for `Python 3.8+` on `mac` and `linux` (#710)
* Explicit `multiprocessing.Manager.shutdown`

Multiprocessing manager is used within eventing core. From doc,
it appears to start a BaseManager which starts a server????
Seriously???? Anyways, using multiprocessing manager is a PITA
and mistake, as it doesn't even give us performance we expect.
Our proxy server can handle more requests than what multiprocess
manager can exchange between processes.

* `--threadless is now ON by default for `Python 3.8+` on `mac` and `linux` environments

* Clarity around why multiprocessing.Manager must be deprecated

* Add `--threaded` flag which can be used to fallback for environments where `--threadless` is now default

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* never used

* Update README

* Use `threaded=True` in tests which were written for threaded model

* Fix issue where sharing manager between global event queue and subscriber can lead to TypeError

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-09 01:34:17 +05:30
Abhinav Singh b830f7b238
Rename `Proxy.initialize` as `FlagParser.initialize` (#696)
* Move Proxy.initialize within FlagParser.initialize.  Also move other staticmethods from within proxy class into utils

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused imports

* Fix `import-outside-toplevel` error

* add `make lib-flake8` and `make lib-mypy` targets

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-07 23:02:34 +05:30
Sviatoslav Sydorenko 05a78567ac
Enable the `add-trailing-comma` pre-commit fixer (#661)
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
2021-11-04 16:58:36 +05:30
Abhinav Singh 0744cd8e7f
Go flagless to allow custom user defined flags. (#452)
* Go flagless to allow custom user defined flags. Fixes #301

* Add --cache-dir flag for cache plugin (when used with on-disk store)

* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand.  This also allows external flags to surface in --help section

* Define --filtered-client-ips flag for FilterByClientIpPlugin
2020-10-14 10:51:56 +05:30
Abhinav Singh c6c09395f9
Share lock to acceptors via pool (#238)
* Move manager initialization outside of top level scope.  Fixes #233

* Share lock to acceptor via pool
2019-12-20 15:13:46 -08:00
Abhinav Singh 269484df2e
Make HTTP handler constructor free of socket file number (#219)
* Refactor into acceptor module

* Add tunnel doc

* Make fileno free

* Autopep8
2019-12-02 15:55:08 -08:00
Abhinav Singh e03cae30d2
Initialize Menubar (#188)
* Initialize MacOS Menubar application

* Dashboard plugin at-least needs a shutdown hook to teardown any thread/processes started by dashboard backend plugin

* Add menu bar icon

* Add respective test directories

* Sync test banners

* Move plugin tests under its own package

* Enable daemon for threads, other this wont shutdown cleanly
2019-11-22 15:18:01 -08:00
Abhinav Singh 3aa1dc2824
Test refactor + Docker image CI (#154)
* Move tests into individual modules too

* Ensure one test class per file

* Fix docker image after refactoring

* Add github actions workflow for building docker image

* Fix image name

* Setup python required for extracting proxy version

* Version will also require deps
2019-10-29 20:41:39 -07:00