Commit Graph

20 Commits

Author SHA1 Message Date
Abhinav Singh 2fa320d03f
Python 3.11 support (#1384)
* Changes for Python 3.11 support

* Updated README.md for versioning info

* Update `httpx==0.27.0` to avoid `cgi` deprecation warning from pytest on Python 3.11

* Make tests work for 3.11

* Declare support for 3.11

* Use 3.11-alpine for Docker images

* Preserve pylint version for `python_version <= 3.10`

* Preserve httpx version for <= 3.10

* `httpx` usage fix in tests for <=3.10

* Adjust pylint and pytest for >= 3.11

* Use 3.11.8, bad-option-value and httpx proxies fix

* tox for 3.11

* Fix for `TOXENV: py`

* -vv for pytest

* Downgrade to `pytest-asyncio==0.21.1`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove asyncio_mode=strict

* try with `pytest-cov==4.1.0` for 3.11

* bump coverage for 3.11

* Try `3.11` in GitHub workflow which installs >3.11.8 unavailable via pyenv yet

* Revert back to `-v`

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-04-13 13:22:25 +05:30
Abhinav Singh 54e74a911f
[isort] Lib modules (#1016)
* isort `proxy.py` main class

* isort init and main

* isort common

* pre-commit fix

* isort dashboard and testing

* isort plugins

* isort core

* Only sort top level http py files

* isort http exception and websocket

* Remove proxy auth plugin from proxy package exports and force discover `PLUGIN_PROXY_AUTH` flags

* isort parser and web server

* no setattr

* isort all

* Enable pre-commit isort hook

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-01-20 15:34:54 +05:30
Abhinav Singh 0ffa7ca9fb
[TlsInterception] Fix broken `ChunkedResponsePlugin` for `Python < 3.10` (#986)
* Add TLS interception integration tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix CI

* unused

* Well 3.9 just worked locally

* Dispatching empty byte to client results in OSError for Python < 3.10

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Uncomment old integration tests

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-01-14 14:35:28 +05:30
Abhinav Singh 46c942f947
Hook `UpstreamConnectionPool` lifecycle within `Threadless` (#921)
* Hook connection pool lifecycle within threadless

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix test

* Fix spell

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-12-29 17:37:15 +05:30
Abhinav Singh f6214a46c9
Move `UpstreamConnectionPool` lifecycle within `Threadless` (#917)
* Tie connection pool into Threadless

* Pass upstream conn pool reference to work instances

* Mark upstream conn pool as optional

* spellcheck

* Fix unused import
2021-12-28 13:51:20 +05:30
Abhinav Singh 2d8385720c
Ensure message for every `HttpProtocolException` raised (#900) 2021-12-22 09:20:06 +05:30
Abhinav Singh da1795d2ed
[ProxyPool] Add support for basic authorization with upstream proxies (#897)
* Use `Url` class to parse proxy pool entries

* Add support for parsing user:pass from raw url bytes

* Add `httpHeaders.PROXY_AUTHORIZATION` headers for upstream proxies

* Add support for httpHeaders enum

* Send base64 encoded proxy authorization header to upstream proxies

* mypy fixes

* Document proxy pool authentication support usage info
2021-12-22 01:48:15 +05:30
Abhinav Singh 90820b01f5
[ProxyPoolPlugin] Avoid remote proxy of private IP requests (#833)
* Avoid proxy of requests to private IP within `ProxyPoolPlugin`

* Fix tests

* spell fix
2021-12-01 05:29:52 +05:30
Abhinav Singh 921f2b56e0
Optimizations & Update Benchmark Results (#832)
* Bail out of lock earlier

* Avoid calling `get_events` for unfinished work tasks.  Use `cached_property` for `HttpParser` optimization

* Add `type: ignore[no-any-return]`, odd scenario

* We dont have to rebuild response packet repeatedly within `WebServerPlugin`

* Parse line and header in one invocation

* Minor optimizations and update benchmark to use `oha` instead of `hey`

* Remove `flask` from benchmark, only benchmark `asgi` or `async` based libraries.  Use `uvicorn` and 10 workers for `blacksheep`.  Use `oha` instead of `hey`

* Add benchmark for `starlette`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add missing dep

* pre-commit

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-12-01 04:18:49 +05:30
Abhinav Singh 8fdddfd199
Acceptors performance (#767)
* Use threads for delegation. Now `run_once` lock before `accept` not `select`

* Add support to use master proxy within proxy pool plugin.  When used, proxy pool plugin will be a no-op for the master node

* Fix acceptor tests now that mask is being used

* Use `cached_property` for web server routes

* Use `select(timeout=1)` otherwise acceptor wont join if total blocking

* mypy, flake, doc spell fixes

* R0205: Class `cached_property` inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
2021-11-20 19:42:38 +05:30
Abhinav Singh 3cfce5265c
Add `TcpUpstreamConnectionHandler` class (#760)
* Add `TcpUpstreamConnectionHandler` which can be used as standalone or as mixin

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* `TcpUpstreamConnectionHandler` is now an abstract class

* Fix mypy

* `nitpick_ignore` the `proxy.core.base.tcp_upstream.TcpUpstreamConnectionHandler` class

* Add mypy exception for now

* Fix flake

* Fix docstring

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-20 00:05:24 +05:30
Abhinav Singh 7f1470e3a9
No abstract method for proxy plugin (#738)
* Remove abstractmethod for proxy plugin base class, remove unused methods from bundled plugins

* Move httpStatusCodes, httpMethods and Url within top-level proxy.http package
2021-11-15 03:17:12 +05:30
Abhinav Singh 541da114c8
Custom Url Parser (#730)
* Custom Url parser for our needs

* lint fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix proxy_pool plugin as scheme can be None if not present in the Url

* Address the ambiguous ipv6:port scenario along with valid cases

* lint checks

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docstring

* Abstract into `http.parser` module

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix #398 HTTP/1.0 related issue

* lint checks

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-12 19:00:19 +05:30
Abhinav Singh 0739db6053
Add `--proxy-pool` flag (#727)
* Add `--proxy-pool` flag

* lint checks
2021-11-12 03:35:23 +05:30
Abhinav Singh ddf90fb6f0
Allow `--plugins` flag to be used multiple times (#725)
* deprecate server_file_or_404

* Optionally compress static content.  Currently only if content length higher than 300

* trailing comma

* Allow `--plugins` flag to be used multiple times

Following are valid invocation:
1) `--plugins A`
2) `--plugins A,B`
3) `--plugins A --plugins B`
4) `--plugins A,B --plugins C`

* mypy

* Flake8

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* correct type

* Add `HttpParser.is_https_tunnel()` utility method

* mypy

* lint checks

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-12 02:46:43 +05:30
Sviatoslav Sydorenko 05a78567ac
Enable the `add-trailing-comma` pre-commit fixer (#661)
Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>
2021-11-04 16:58:36 +05:30
Abhinav Singh d4ee4fa15c
Async proxy pool, Event manager, Custom access log, Expose loop to plugins (#645)
* Async proxy pool

* Async proxy pool

* Late upstream initialization and exception guards

* Close upstream proxy connection on client connection close

* Refactor into EventManager

* Fix tests accounting in the event manager

* Ensure each process initializes logger

* pragma no cover

* Teardown connection when proxy pool upstream proxy closes

* Add ability to customize access log format and add additional context to it

* Maintain total size for response bytes in access logs

* Fix tests broken due to new plugin methods missing mock

* Update pubsub_eventing to use EventManager to avoid entire bootstrapping step
2021-11-01 01:19:19 +05:30
Abhinav Singh 7448c44cc6
v3.4.0 (#638)
* Build docker from 3.10-alpine

* Bump version to 3.4.0

* Add instructions for how to run dashboard

* Order of menu

* Override dashboard png path until submitted

* Add some doc string for top-level Proxy class.  Also some TODOs and warnings regarding PID file overwrite

* Allow HttpProxyBasePlugin implementations to register custom descriptors for read/write events

* Remove hardcoded adblock regex into json config. Update upstream filter to block facebook, not google

* ProxyPoolPlugin and ReverseProxyPlugin must now be updated to use get/read/write descriptor APIs

* Add get/read/write descriptor API for HttpWebServerBasePlugin too

* Surface actual listening port via flags.port
2021-10-30 04:32:05 +05:30
pva 27e3cadf03
Fix ProxyPoolPlugin.rebuild_original_path 2020-02-01 10:16:19 -08:00
Abhinav Singh ba5c84dfbd
Proxy Pool Plugin (#228)
* Add proxy pool example. See #226

* Add ProxyPoolPlugin to doc
2019-12-09 19:38:49 -08:00