Rooba
/
proxy.py
mirror of https://github.com/abhinavsingh/proxy.py.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
proxy.py/README.md

148 lines
8.6 KiB
Markdown
Raw Normal View History

Link Travis Board for build status image
2019-08-12 05:41:39 +00:00
[![Proxy.Py](ProxyPy.png)](https://github.com/abhinavsingh/proxy.py)
updated README.md
2013-08-31 09:19:22 +00:00
Add plugin usage instruction
2019-09-21 22:04:44 +00:00
[![License](https://img.shields.io/github/license/abhinavsingh/proxy.py.svg)](https://opensource.org/licenses/BSD-3-Clause)
Make docker v1.0.0 release (#82) * Make docker v1.0.0 release * 👍 * 👍 📱 💻 💡 * 👍 📱 💻 💡
2019-09-23 07:30:52 +00:00
[![PyPi Downloads](https://img.shields.io/pypi/dm/proxy.py.svg?color=green)](https://pypi.org/project/proxy.py/)
[![Docker Pulls](https://img.shields.io/docker/pulls/abhinavsingh/proxy.py?color=green)](https://hub.docker.com/r/abhinavsingh/proxy.py)
Use Pipe instead of Queue when passing sockets (#69) * Use pipes instead of queues. Possible solution for #68 * Handle ConnectionRefusedError * Close corresponding pipes on shutdown * Abstract classes for plugins. * Add github workflows * Use pytest for github workflow * Add Windows/Mac workflows * Use os.matrix * 3.8 and 3.9 probably dont exists yet * Do not fail-fast, also python 3.5 seems to be throwing syntax error for typing * autopep8 * Disable windows-latest workflow which seems to hang
2019-09-17 03:18:14 +00:00
[![Build Status](https://travis-ci.org/abhinavsingh/proxy.py.svg?branch=develop)](https://travis-ci.org/abhinavsingh/proxy.py/)
Make docker v1.0.0 release (#82) * Make docker v1.0.0 release * 👍 * 👍 📱 💻 💡 * 👍 📱 💻 💡
2019-09-23 07:30:52 +00:00
[![No Dependencies](https://img.shields.io/static/v1?label=dependencies&message=none&color=green)](https://github.com/abhinavsingh/proxy.py)
Use Pipe instead of Queue when passing sockets (#69) * Use pipes instead of queues. Possible solution for #68 * Handle ConnectionRefusedError * Close corresponding pipes on shutdown * Abstract classes for plugins. * Add github workflows * Use pytest for github workflow * Add Windows/Mac workflows * Use os.matrix * 3.8 and 3.9 probably dont exists yet * Do not fail-fast, also python 3.5 seems to be throwing syntax error for typing * autopep8 * Disable windows-latest workflow which seems to hang
2019-09-17 03:18:14 +00:00
[![Coverage](https://coveralls.io/repos/github/abhinavsingh/proxy.py/badge.svg?branch=develop)](https://coveralls.io/github/abhinavsingh/proxy.py?branch=develop)
Make docker v1.0.0 release (#82) * Make docker v1.0.0 release * 👍 * 👍 📱 💻 💡 * 👍 📱 💻 💡
2019-09-23 07:30:52 +00:00
[![Tested With Android](https://img.shields.io/static/v1?label=tested%20with&message=Android%20%F0%9F%93%B1&color=brightgreen)](https://www.android.com/)
[![Tested With Android Emulator](https://img.shields.io/static/v1?label=tested%20with&message=Android%20Emulator%20%F0%9F%93%B1&color=brightgreen)](https://developer.android.com/studio/run/emulator-networking.html#proxy)
[![Tested With iOS](https://img.shields.io/static/v1?label=tested%20with&message=iOS%20%F0%9F%93%B1&color=brightgreen)](https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html)
[![Tested With iOS Simulator](https://img.shields.io/static/v1?label=tested%20with&message=iOS%20Simulator%20%F0%9F%93%B1&color=brightgreen)](https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html)
[![Tested With MacOS](https://img.shields.io/static/v1?label=tested%20with&message=mac%20OS%20%F0%9F%92%BB&color=brightgreen)](https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html)
[![Tested With Ubuntu](https://img.shields.io/static/v1?label=tested%20with&message=Ubuntu%20%F0%9F%96%A5&color=brightgreen)](https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html)
[![Tested With Windows](https://img.shields.io/static/v1?label=tested%20with&message=Windows%20%F0%9F%92%BB&color=brightgreen)](https://developer.apple.com/library/archive/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html)
[![Maintenance](https://img.shields.io/static/v1?label=maintained%3F&message=yes&color=green)](https://gitHub.com/abhinavsingh/proxy.py/graphs/commit-activity)
[![Ask Me Anything](https://img.shields.io/static/v1?label=need%20help%3F&message=ask&color=green)](https://twitter.com/imoracle)
[![Contributions Welcome](https://img.shields.io/static/v1?label=contributions&message=welcome%20%F0%9F%91%8D&color=green)](https://github.com/abhinavsingh/proxy.py/issues)
Add plugin usage instruction
2019-09-21 22:04:44 +00:00
[![Python 3.5](https://img.shields.io/badge/python-3.5-blue.svg)](https://www.python.org/downloads/release/python-350/)
[![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/)
[![Python 3.7](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/downloads/release/python-370/)
Make docker v1.0.0 release (#82) * Make docker v1.0.0 release * 👍 * 👍 📱 💻 💡 * 👍 📱 💻 💡
2019-09-23 07:30:52 +00:00
[![Checked with mypy](https://img.shields.io/static/v1?label=mypy&message=checked&color=blue)](http://mypy-lang.org/)
Add Travis build status
2019-02-09 04:25:09 +00:00
updated README.md
2013-08-31 09:19:22 +00:00
Features
--------
Add plugin usage instruction
2019-09-21 22:04:44 +00:00
- Lightweight
- Distributed as a single file module `~50KB`
Use ipaddress type and avoid explicit Connection: close header to upstream (#76) * Fixes #71 and address some of #75 * Fix tests as IPV6 is default now * Add comments and raise from e
2019-09-22 02:20:27 +00:00
- Uses only `~5-20MB` RAM
Add plugin usage instruction
2019-09-21 22:04:44 +00:00
- No external dependency other than standard Python library
- Programmable
- Optionally enable builtin Web Server
- Customize proxy and http routing via [plugins](https://github.com/abhinavsingh/proxy.py/blob/develop/plugin_examples.py)
- Enable plugin using command line option e.g. `--plugins plugin_examples.SaveHttpResponses`
- Plugin API is currently in development state, expect breaking changes.
- Supports `http`, `https`, `http2` and `websockets` request proxy
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
- Optimized for large file uploads and downloads
Avoid setting open file limits on Windows OS. Python resource module is unavailable on Windows OS.
2018-12-14 13:14:56 +00:00
- IPv4 and IPv6 support
- Basic authentication support
Link Travis Board for build status image
2019-08-12 05:41:39 +00:00
- Can serve a [PAC (Proxy Auto-configuration)](https://en.wikipedia.org/wiki/Proxy_auto-config) file
updated README.md
2013-08-31 09:19:22 +00:00
Install
-------
Use Pipe instead of Queue when passing sockets (#69) * Use pipes instead of queues. Possible solution for #68 * Handle ConnectionRefusedError * Close corresponding pipes on shutdown * Abstract classes for plugins. * Add github workflows * Use pytest for github workflow * Add Windows/Mac workflows * Use os.matrix * 3.8 and 3.9 probably dont exists yet * Do not fail-fast, also python 3.5 seems to be throwing syntax error for typing * autopep8 * Disable windows-latest workflow which seems to hang
2019-09-17 03:18:14 +00:00
#### Stable version
updated README.md
2013-08-31 09:19:22 +00:00
Add Travis build status
2019-02-09 04:25:09 +00:00
$ pip install --upgrade proxy.py
updated README.md
2013-08-31 09:19:22 +00:00
Use Pipe instead of Queue when passing sockets (#69) * Use pipes instead of queues. Possible solution for #68 * Handle ConnectionRefusedError * Close corresponding pipes on shutdown * Abstract classes for plugins. * Add github workflows * Use pytest for github workflow * Add Windows/Mac workflows * Use os.matrix * 3.8 and 3.9 probably dont exists yet * Do not fail-fast, also python 3.5 seems to be throwing syntax error for typing * autopep8 * Disable windows-latest workflow which seems to hang
2019-09-17 03:18:14 +00:00
#### Development version
$ pip install git+https://github.com/abhinavsingh/proxy.py.git@develop
#### Docker image
Release docker container and prep for v0.4
2019-02-09 05:13:34 +00:00
Make docker v1.0.0 release (#82) * Make docker v1.0.0 release * 👍 * 👍 📱 💻 💡 * 👍 📱 💻 💡
2019-09-23 07:30:52 +00:00
$ docker run -it -p 8899:8899 --rm abhinavsingh/proxy.py:v1.0.0
By default `docker` binary is started with following flags:
--hostname 0.0.0.0 --port 8899 --ipv4
To override input flags, start docker image as follows.
For example, to check `proxy.py --version`:
$ docker run -it \
-p 8899:8899 \
--entrypoint /app/proxy.py \
--rm abhinavsingh/proxy.py:v1.0.0 \
--version
[![WARNING](https://img.shields.io/static/v1?label=MacOS&message=warning&color=red)](https://github.com/moby/vpnkit/issues/469)
`docker` image is currently broken on `macOS` due to incompatibility with [vpnkit](https://github.com/moby/vpnkit/issues/469).
Release docker container and prep for v0.4
2019-02-09 05:13:34 +00:00
updated README.md
2013-08-31 09:19:22 +00:00
Usage
-----
```
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
$ proxy.py -h
usage: proxy.py [-h] [--backlog BACKLOG] [--basic-auth BASIC_AUTH]
Provision to specify custom buffer sizes. Now --server-recvbuf-size and --client-recvbuf-size flags can be used for customizing maximum recv buffer sizes. Bumping these values will result in higher RAM consumption during download / upload operations but will provide better download / upload speeds / throughputs and lower CPU utilization. Fixes #22
2018-12-13 21:08:08 +00:00
[--client-recvbuf-size CLIENT_RECVBUF_SIZE]
Add --disable-headers option. Fixes #31
2019-09-02 22:58:37 +00:00
[--disable-headers DISABLE_HEADERS] [--disable-http-proxy]
[--hostname HOSTNAME] [--ipv4] [--enable-web-server]
[--log-level LOG_LEVEL] [--log-file LOG_FILE]
[--log-format LOG_FORMAT] [--num-workers NUM_WORKERS]
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
[--open-file-limit OPEN_FILE_LIMIT] [--pac-file PAC_FILE]
Add --pid-file option to dump parent process ID to a file. Rename to --disable-http-proxy. Is more appropriate since proxy is always enabled. unittest.skipIf right guards for travis
2019-08-26 16:52:43 +00:00
[--pac-file-url-path PAC_FILE_URL_PATH] [--pid-file PID_FILE]
[--plugins PLUGINS] [--port PORT]
[--server-recvbuf-size SERVER_RECVBUF_SIZE] [--version]
updated README.md
2013-08-31 09:19:22 +00:00
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
proxy.py v1.0.0
updated README.md
2013-08-31 09:19:22 +00:00
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
optional arguments:
updated README.md
2013-08-31 09:19:22 +00:00
-h, --help show this help message and exit
Update README to reflect basic auth support. Also refactor proxy exception handling code.
2018-12-13 17:01:56 +00:00
--backlog BACKLOG Default: 100. Maximum number of pending connections to
proxy server
--basic-auth BASIC_AUTH
Default: No authentication. Specify colon separated
user:password to enable basic authentication.
Provision to specify custom buffer sizes. Now --server-recvbuf-size and --client-recvbuf-size flags can be used for customizing maximum recv buffer sizes. Bumping these values will result in higher RAM consumption during download / upload operations but will provide better download / upload speeds / throughputs and lower CPU utilization. Fixes #22
2018-12-13 21:08:08 +00:00
--client-recvbuf-size CLIENT_RECVBUF_SIZE
Add option (--log-file) to redirect logs to a file
2019-08-26 16:37:25 +00:00
Default: 1 MB. Maximum amount of data received from
Provision to specify custom buffer sizes. Now --server-recvbuf-size and --client-recvbuf-size flags can be used for customizing maximum recv buffer sizes. Bumping these values will result in higher RAM consumption during download / upload operations but will provide better download / upload speeds / throughputs and lower CPU utilization. Fixes #22
2018-12-13 21:08:08 +00:00
the client in a single recv() operation. Bump this
value for faster uploads at the expense of increased
RAM.
Add --disable-headers option. Fixes #31
2019-09-02 22:58:37 +00:00
--disable-headers DISABLE_HEADERS
Default: None. Comma separated list of headers to
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
remove before dispatching client request to upstream
Add --disable-headers option. Fixes #31
2019-09-02 22:58:37 +00:00
server.
--disable-http-proxy Default: False. Whether to disable
proxy.HttpProxyPlugin.
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
--hostname HOSTNAME Default: 127.0.0.1. Server IP address.
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
--ipv4 Whether to listen on IPv4 address. By default server
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
only listens on IPv6.
Add --version flag also used by Makefile for releases.
2019-08-24 17:23:30 +00:00
--enable-web-server Default: False. Whether to enable
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
proxy.HttpWebServerPlugin.
updated README.md
2013-08-31 09:19:22 +00:00
--log-level LOG_LEVEL
Update README.md
2019-07-20 19:17:01 +00:00
Valid options: DEBUG, INFO (default), WARNING, ERROR,
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
CRITICAL. Both upper and lowercase values are allowed.
You may also simply use the leading character e.g.
--log-level d
Add option (--log-file) to redirect logs to a file
2019-08-26 16:37:25 +00:00
--log-file LOG_FILE Default: sys.stdout. Log file destination.
Core changes: 1. Rename HttpServer to MultiCoreRequestDispatcher, there wasn't anything Http about that class, all Http implementation happens in HttpProxy class 2. PY 2.7 DEPRECATION ERRORS 3. Add --log-format cli option 4. Remove _ from HttpProxy private methods, refactor it's core for clarify (remove redundant close() and other details)
2019-08-20 05:59:11 +00:00
--log-format LOG_FORMAT
Log format for Python logger.
Add --version flag also used by Makefile for releases.
2019-08-24 17:23:30 +00:00
--num-workers NUM_WORKERS
Defaults to number of CPU cores.
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
--open-file-limit OPEN_FILE_LIMIT
Default: 1024. Maximum number of files (TCP
connections) that proxy.py can open concurrently.
Update README.md
2019-07-20 19:17:01 +00:00
--pac-file PAC_FILE A file (Proxy Auto Configuration) or string to serve
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
when the server receives a direct file request. Using
this option enables proxy.HttpWebServerPlugin.
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
--pac-file-url-path PAC_FILE_URL_PATH
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
Default: /. Web server path to serve the PAC file.
Add --pid-file option to dump parent process ID to a file. Rename to --disable-http-proxy. Is more appropriate since proxy is always enabled. unittest.skipIf right guards for travis
2019-08-26 16:52:43 +00:00
--pid-file PID_FILE Default: None. Save parent process ID to a file.
Per worker queue instead of shared one. 1. MultiCoreDispatcher now dispatches request in round robin fashion. 2. Added Python3 static type hinting add Google Cloud Run Button Preserve proxy.py name as otherwise it breaks module system. Just provide a custom service name to fix Google Cloud Run issue Custom service name for Google Cloud Run. Specify port in app.json as required by Google Cloud Run Go cannot unmarshall int :) Port desc Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile Remove cloudrun integration. While proxy.py runs successfully within the container, gvisor limitations doesn't allow proxy.py to accept client connections. Specifically membarrier syscall isn't allowed. Update README.md
2019-08-23 00:09:35 +00:00
--plugins PLUGINS Comma separated plugins
--port PORT Default: 8899. Server port.
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
--server-recvbuf-size SERVER_RECVBUF_SIZE
Add option (--log-file) to redirect logs to a file
2019-08-26 16:37:25 +00:00
Default: 1 MB. Maximum amount of data received from
Pre-spawn worker processes to utilize all cores.
2019-07-20 19:47:16 +00:00
the server in a single recv() operation. Bump this
value for faster downloads at the expense of increased
RAM.
Address mypy errors. Unfortunately collections.namedtuple isn't supported by mypy and most likely never will be. Idea will be to use Typing.NamedTuple but this require some code refactoring.
2019-09-14 04:18:05 +00:00
--version, -v Prints proxy.py version.
updated README.md
2013-08-31 09:19:22 +00:00
Update README.md
2019-07-20 19:17:01 +00:00
Proxy.py not working? Report at:
updated README.md
2013-08-31 09:19:22 +00:00
https://github.com/abhinavsingh/proxy.py/issues/new
```