Commit Graph

167 Commits

Author SHA1 Message Date
Abhinav Singh 92142a70d9
Add support for TLS/SSL interception and HTTPS server (#86)
* Handle OSError thrown when curl --proxy-cacert flag is invalid

* Add server-cert

* Dockerfile dont need --ipv4 since now its auto-detected and deprecated

* Add make server-cert info

* Add support for HTTPS interception.

1) Start as

    proxy.py --ca-key-file ca-key.pem --ca-cert-file ca-cert.pem --ca-signing-key-file ca-signing-key.pem

2) Test using curl

    curl -v -L -x proxy.py:8899 --cacert ca-cert.pem https://google.com

Now proxy.py is able to decrypt HTTPS traffic between curl and
google.com.  Generated certificates are stored under ~/.proxy.py
directory.

* Add comments for why interception do not work if client is communicating with proxy.py over https

* Fix tests

* Exit with message when trying to https intercept and also serve over https

* lint

* Disable OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1, OP_NO_TLSv1_1 when wrapping upstream server socket

* Ensure client.conn.shutdown is called if necessary.

* Add mypy check for plugin_examples.py

* Exception handling when client sends invalid requests

* Verify plugin_examples

* mypy compat

* HttpRequestRejected
2019-09-24 19:02:57 -07:00
pyup.io bot 8ecb76da2f Update twine from 1.15.0 to 2.0.0 (#88) 2019-09-24 09:13:11 -07:00
Abhinav Singh 397f92e561
Can serve proxy traffic over HTTPS (#85)
* Can serve proxy traffic over HTTPS.

Pass --certfile and --keyfile arguments to enable SSL/TLS layer.
Example:

1) Start proxy.py as

./proxy.py --certfile cert.pem --keyfile key.pem

2) Then verify via curl

curl -v -x https://localhost:8899 --proxy-cacert cert.pem http://httpbin.org/get

* Add --ignore-missing-imports for missing multiprocessing.reduction module

* mock socket.fromfd to avoid sys calls

* Add HTTPS info

* Add TLS encryption

* Add TLS encryption
2019-09-23 17:38:10 -07:00
Abhinav Singh 2b8fd53f15
Make docker v1.0.0 release (#82)
* Make docker v1.0.0 release

* 👍

* 👍 📱 💻 💡

* 👍 📱 💻 💡
2019-09-23 00:30:52 -07:00
Abhinav Singh ae1f1d654c
mypy coverage (#78)
* mypy coverage

* First stab at mypy --strict complaince

* mypy --strict compliance.

* Run mypy during Quality Check phase
2019-09-22 21:11:55 -07:00
Abhinav Singh 0fa3a3f4d7
Resolve issues reported by Bandit (#77)
* Resolve issues reported by Bandit

* Fix tests
2019-09-21 23:50:31 -07:00
Abhinav Singh 5bdfd20ce8
Use ipaddress type and avoid explicit Connection: close header to upstream (#76)
* Fixes #71 and address some of #75

* Fix tests as IPV6 is default now

* Add comments and raise from e
2019-09-21 19:20:27 -07:00
Abhinav Singh 5d4f735415 Add plugin usage instruction 2019-09-21 15:04:44 -07:00
pyup.io bot 15796cf5de Initial Update (#74)
* Update coverage from 4.5.2 to 4.5.4

* Update twine from 1.12.1 to 1.15.0

* Update pytest from 5.1.2 to 5.1.3
2019-09-21 14:51:51 -07:00
Abhinav Singh 3bc81c3e75
Http parser fix for state completion (#73)
* Fix HttpParser to correctly reach COMPLETE state.

Fixes #70, see test case test_pip_connect

* Remove redundant logic.

* Lint fixes
2019-09-21 13:58:04 -07:00
Abhinav Singh 5eb4d8a97c
Use Pipe instead of Queue when passing sockets (#69)
* Use pipes instead of queues.

Possible solution for #68

* Handle ConnectionRefusedError

* Close corresponding pipes on shutdown

* Abstract classes for plugins.

* Add github workflows

* Use pytest for github workflow

* Add Windows/Mac workflows

* Use os.matrix

* 3.8 and 3.9 probably dont exists yet

* Do not fail-fast, also python 3.5 seems to be throwing syntax error for typing

* autopep8

* Disable windows-latest workflow which seems to hang
2019-09-16 20:18:14 -07:00
Abhinav Singh 939303452a HttpParser fix for responses 2019-09-14 00:23:39 -07:00
Abhinav Singh a7d0a30278 Android Emulator fixes #65 2019-09-13 23:08:12 -07:00
Abhinav Singh e4ba6b16cd Address mypy errors.
Unfortunately collections.namedtuple isn't supported by mypy and most
likely never will be. Idea will be to use Typing.NamedTuple but this
require some code refactoring.
2019-09-13 21:18:05 -07:00
Abhinav Singh 80c73a4798 Do not store raw response bytes for CONNECT requests.
Fixes #67 and addresses #66 too.
2019-09-13 11:03:26 -07:00
Abhinav Singh 3e92faba32 Necessary productionization tests. 2019-09-03 12:54:47 -07:00
Abhinav Singh ef4003c1be Merge branch 'develop' of github.com:abhinavsingh/proxy.py into develop 2019-09-03 09:54:09 -07:00
Abhinav Singh 81b678f223 Update LICENSE 2019-09-03 09:53:14 -07:00
Abhinav Singh d387013a3f Merge branch 'master' into develop 2019-09-03 09:51:36 -07:00
Abhinav Singh 7e0e379a76 Merge branch 'develop' of github.com:abhinavsingh/proxy.py into develop 2019-09-03 08:58:09 -07:00
Abhinav Singh c80901689a Add Procfile.
Several forks have introduced a Procfile. It doesn't hurt to have one in
the repo itself.
2019-09-03 08:57:20 -07:00
Abhinav Singh cd10b9aa24
Update LICENSE 2019-09-02 22:55:34 -07:00
Abhinav Singh 95671e4905 Address ipv6 parsing from Http requests #65 2019-09-02 22:08:54 -07:00
Abhinav Singh c3cd9be816 Add --disable-headers option. Fixes #31 2019-09-02 15:58:37 -07:00
Abhinav Singh 8489e8bc2f Fix default listen on IPV6 for linux systems.
Closes #59
2019-09-02 11:20:26 -07:00
Abhinav Singh 3bf0f0754b v1.0 2019-08-26 10:54:19 -07:00
Abhinav Singh d0f696212e Add --pid-file option to dump parent process ID to a file.
Rename to --disable-http-proxy.

Is more appropriate since proxy is always enabled.

unittest.skipIf right guards for travis
2019-08-26 10:54:17 -07:00
Abhinav Singh a5a0149402 Add option (--log-file) to redirect logs to a file 2019-08-26 10:54:17 -07:00
Abhinav Singh fcfbec4760 Add coveralls integration
No use mirrors

Enforce coverage3

Add coverage badge and use python -m to invoke coverage

Enforce pip3, remove SyntaxError reported line

Use develop branch for coverage status
2019-08-26 10:54:17 -07:00
Abhinav Singh 3a371f2d6f Add tests for proxy.main 2019-08-26 10:54:17 -07:00
Abhinav Singh 19d47ad792 Create separate base plugin HttpProxyBasePlugin for proxy handling only
Disable open file limit tests for Windows
2019-08-26 10:54:14 -07:00
Abhinav Singh eeb983f988 Add tests for HttpRequestRejected responses and open file limit setter. 2019-08-24 11:31:55 -07:00
Abhinav Singh 8ba63b1821 Add --version flag also used by Makefile for releases. 2019-08-24 10:23:30 -07:00
Abhinav Singh e463d428b9 Enable WebServer plugin when --pac_file serving is requested.
1. Also binary read pac file instead of str
2. Return 404 for unhandled WebServer requests
2019-08-24 10:11:57 -07:00
Abhinav Singh 2c239a1d25 Merge branch 'develop' of github.com:abhinavsingh/proxy.py into develop 2019-08-24 09:51:18 -07:00
Abhinav Singh 5343350c3c
Merge pull request #62 from abhinavsingh/plugins
Proxy Plugins
2019-08-24 09:42:51 -07:00
Abhinav Singh 4019e95850 Per worker queue instead of shared one.
1. MultiCoreDispatcher now dispatches request in round robin fashion.
2. Added Python3 static type hinting

add Google Cloud Run Button

Preserve proxy.py name as otherwise it breaks module system.  Just provide a custom service name to fix Google Cloud Run issue

Custom service name for Google Cloud Run.

Specify port in app.json as required by Google Cloud Run

Go cannot unmarshall int :)

Port desc

Looks like PORT cannot be overridden, for now use port 8080 in Dockerfile

Remove cloudrun integration.

While proxy.py runs successfully within the container, gvisor
limitations doesn't allow proxy.py to accept client connections.
Specifically membarrier syscall isn't allowed.

Update README.md
2019-08-24 09:31:56 -07:00
Abhinav Singh 6ea42b0dd9 Convert HttpProxy and HttpWebServer itself into plugins.
Load plugins during test execution

Further decouple proxy/webserver logic outside of HttpProtocolHandler.

Per connection plugin instances to avoid locks

Handle BrokenPipeError and teardown if read_from_descriptors return True
2019-08-24 09:31:53 -07:00
Abhinav Singh 8732cb7151 Refactor HttpProxy.run
Fix tests for py2.7 which require external mock module

Disable tests which require listening on sockets for Travis.

Make travis make free

set TESTING_ON_TRAVIS=1 for windows
2019-08-24 09:31:50 -07:00
Abhinav Singh 6b49de2ce6 Core changes:
1. Rename HttpServer to MultiCoreRequestDispatcher, there wasn't
anything Http about that class, all Http implementation happens in
HttpProxy class
2. PY 2.7 DEPRECATION ERRORS
3. Add --log-format cli option
4. Remove _ from HttpProxy private methods, refactor it's core for
clarify (remove redundant close() and other details)
2019-08-24 09:31:50 -07:00
Abhinav Singh 03104cc54d Add TcpServer tests and make accept non-blocking.
Non-blocking accepts helps specially in tests to abort the test server
gracefully.

Add ipv6 tcp server client tests

Add tests for TcpServer and HttpServer

Add HttpProxyConfig object to avoid unnecessary arg passing.
2019-08-24 09:31:45 -07:00
Abhinav Singh c8fabb86c4 Camel case consistency and more tests.
Add plugins to proxy.py for programmable proxy.

As requested in https://github.com/abhinavsingh/proxy.py/issues/30 and
other mediums, introduce proxy.py plugins.  plugin_examples.py provides
examples for how to use them.

Add capability for plugins to simply ProxyRejectRequest.
2019-08-24 09:30:59 -07:00
Abhinav Singh d9946015a7 Camel case consistency 2019-08-12 09:45:27 -07:00
Abhinav Singh 60e1ac1bc2 Link Travis Board for build status image 2019-08-11 22:41:39 -07:00
Abhinav Singh 824ba57589 Disable tests for Python 2.7 version 2019-08-11 21:47:24 -07:00
Abhinav Singh 653163532c Remove -v flag 2019-08-11 21:43:26 -07:00
Abhinav Singh be7235bf0d Add tests for worker and main 2019-08-11 21:02:51 -07:00
Abhinav Singh 4c09a677a3 Add proxy.py banner image 2019-08-11 10:43:59 -07:00
Abhinav Singh 91be448045 Only run if Python 3+.
Some users are already being effected by this, however master branch is
still stable.  Python 2.7 users can continue using the master branch or
simply install via pip.  This change also targets issue#58
2019-08-11 01:36:24 -07:00
Abhinav Singh 271352c9dc
Merge pull request #57 from abhinavsingh/refactor
Refactor
2019-07-26 09:28:33 -07:00