Commit Graph

22 Commits

Author SHA1 Message Date
Abhinav Singh ac29e34137
Add CLI usage for pki.py and update Makefile (#254)
* Add CLI usage for pki.py

* Bump to 2.1.0

* Replace direct openssl invocation with pki utility

* Bolder

* Ordering and version in README

* Refine help
2020-01-06 19:51:18 -08:00
Abhinav Singh 63e6d22566
Optionally initialize manager in main thread and use the same for EventQueue initialization (#239) 2019-12-20 17:10:35 -08:00
Abhinav Singh 625dc4d3e5
Move manager initialization outside of top level scope. Fixes #233 (#236) 2019-12-20 14:01:26 -08:00
Abhinav Singh 54d2a1cc32
Packaging (#210)
* Move docker installation steps above

* Try brewing with virtualenv

* depends on python

* Update homebrew formula for stable release

* Just test brewing on latest python
2019-12-01 20:08:50 -08:00
Abhinav Singh 3bd61e0389
Add tox.ini (#208) 2019-12-01 12:57:21 -08:00
Abhinav Singh 654696d2f9
Test Core Eventing (#205)
* Add core event tests

* Update .gitignore with coverage

* Add shortlink gif

* Add event dispatcher test

* Test event subscriber
2019-11-30 22:04:43 -08:00
Abhinav Singh 0cc4e5e625
Add private / public key generation utils which comply with new requirements on Mac OS 10.15 (#160)
* Add utilities to generate private key and public keys with alternate cnames

* Add separate package proxy.py-plugins, fixes #156

* Generate certificates to comply with Mac requirements.

* Add utility for CSR generation and signing

* Fixes #161

* Add initial pki tests
2019-11-11 16:17:13 -08:00
Abhinav Singh 183d03b866
Separate packages for Dashboard (#157)
* Refactor Makefile and add dashboard setup.py

* Package dashboard as proxy.py-dashboard pip package

* Give dashboard releases its own version

* Fix lib-package reference
2019-11-06 01:33:12 -08:00
Abhinav Singh 3aa1dc2824
Test refactor + Docker image CI (#154)
* Move tests into individual modules too

* Ensure one test class per file

* Fix docker image after refactoring

* Add github actions workflow for building docker image

* Fix image name

* Setup python required for extracting proxy version

* Version will also require deps
2019-10-29 20:41:39 -07:00
Abhinav Singh e14548252c
Proxy.py Dashboard (#141)
* Remove redundant variables

* Initialize frontend dashboard app (written in typescript)

* Add a WebsocketFrame.text method to quickly build a text frame raw packet, also close connection for static file serving, atleast Google Chrome seems to hang up instead of closing the connection

* Add read_and_build_static_file_response method for reusability in plugins

* teardown websocket connection when opcode CONNECTION_CLOSE is received

* First draft of proxy.py dashboard

* Remove uglify, obfuscator is superb enough

* Correct generic V

* First draft of dashboard

* ProtocolConfig is now Flags

* First big refactor toward no-single-file-module

* Working tests

* Update dashboard for refactored imports

* Remove proxy.py as now we can just call python -m proxy -h

* Fix setup.py for refactored code

* Banner update

* Lint check

* Fix dashboard static serving and no UNDER_TEST constant necessary

* Add support for plugin imports when specified in path/to/module.MyPlugin

* Update README with instructions to run proxy.py after refactor

* Move dashboard under /dashboard path

* Rename to devtools.ts

* remove unused

* Update github workflow for new directory structure

* Update test command too

* Fix coverage generation

* *.py is an invalid syntax on windows

* No * on windows

* Enable execution via github zip downloads

* Github Zip downloads cannot be executed as Github puts project under a folder named after Github project, this breaks python interpreter expectation of finding a __main__.py in the root directory

* Forget zip runs for now

* Initialize ProxyDashboard on page load rather than within typescript i.e. on script load

* Enforce eslint with standard style

* Add .editorconfig to make editor compatible with various style requirements (Makefile, Typescript, Python)

* Remove extra empty line

* Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected

* Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected

* Fix tests

* Move common code under common sub-module

* Move flags under common module

* Move acceptor under core

* Move connection under core submodule

* Move chunk_parser under http

* Move http_parser as http/parser

* Move http_methods as http/methods

* Move http_proxy as http/proxy

* Move web_server as http/server

* Move status_codes as http/codes

* move websocket as http/websocket

* Move exception under http/exception, also move http/proxy exceptions under http/exceptions

* move protocol_handler as http/handler

* move devtools as http/devtools

* Move version under common/version

* Lifecycle if now core Event

* autopep8

* Add core event queue

* Register / unregister handler

* Enable inspection support for frontend dashboard

* Dont give an illusion of exception for HttpProtocolExceptions

* Update readme for refactored codebase

* DictQueueType everywhere

* Move all websocket API related code under WebsocketApi class

* Inspection enabled on tab switch.

1. Additionally now acceptors are assigned an int id.
2. Fix tests to match change in constructor.

* Corresponding ends of the work queues can be closed immediately.

Since work queues between AcceptorPool and Acceptor process is used only
once, close corresponding ends asap instead of at shutdown.

* No need of a manager for shared multiprocess Lock.

This unnecessarily creates additional manager process.

* Move threadless into its own module

* Merge acceptor and acceptor_pool tests

* Defer os.close

* Change content display with tab clicks.

Also ensure relay manager shutdown.

* Remove --cov flags

* Use right type for SyncManager

* Ensure coverage again

* Print help to discover flags, --cov certainly not available on Travis for some reason

* Add pytest-cov to requirements-testing

* Re-add windows on .travis also add changelog to readme

* Use 3.7 and no pip upgrade since it fails on travis windows

* Attempt to fix pip install on windows

* Disable windows on travis, it fails and uses 3.8.  Try reporting coverage from github actions

* Move away from coveralls, use codecov

* Codecov app installation either didnt work or token still needs to be passed

* Remove travis CI

* Use https://github.com/codecov/codecov-action for coverage uploads

* Remove run codecov

* Ha, codecov action only works on linux, what a mess

* Add cookie.js though unable to use it with es5/es6 modules yet

* Enable testing for python 3.8 also Build dashboard during testing

* No python 3.8 on github actions yet

* Autopep8

* Add separate workflows for library (python) and dashboard (node) app

* Type jobs not job

* Add checkout

* Fix parsing node version

* Fix dashboard build on windows

* Show codecov instead of coveralls
2019-10-28 14:57:33 -07:00
Abhinav Singh a1bb659488
Threadless execution using coroutines (#134)
* Workers need not register/unregister sock for every loop

* No need of explicit socket.settimeout(0) which is same as socket.setblocking(False)

* Remove settimeout assertion

* Only store sender side of Pipe().  Also ensure both end of the Pipe() are closed on shutdown

* Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes

* Use time.time throughout.  Remove incomplete test_cache_responses_plugin to avoid resource leak in tests

* Remove unused

* Wrap selector register/unregister within a context manager

* Refactor in preparation of threadless request handling

* MyPy generator fix

* Add --threadless flag

* Internally call them acceptors

* Internally use acceptors

* Add Threadless class.

Also no need to pass family over pipe to acceptors.

* Make threadless work for a single client :)

* Threadless is soon be our default

* Close client queue

* Use context manager for register/unregister

* Fix Acceptor tests broken after refactoring

* Use asyncio tasks to invoke ProtocolHandle.handle_events

This gives all client threads a chance to respond without
waiting for other handlers to return.

* Explicitly initialize event loop per Threadless process

* Mypy fixes

* Add ThreadlessWork abstract class implemented by ProtocolHandler

* Add benchmark.py

Avoid TIME_WAIT by properly shutting down the connection.

* Add benchmark.py as part of testing workflow

* When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state

* MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence

* Unwrap if wrapped before shutdown

* Unwrap if wrapped before shutdown

* socket.SHUT_RDWR will cause leaks

* MyPy

* Add instructions for monitor.sh

* Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized

* Add Fast & Scalable section

* Update internal classes section

* Dont print out local dir path in help text :)

* Refactor

* Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang

* Add chrome_with_proxy.sh helper script

* Handle OSError during client.flush which can happen due to invalid protocol type for socket error

* Remove redundant e

* Add classmethods to quickly construct a parser object

* Don't raise from TcpConnection abstract class.

This allows both client/socket side of communication to handle
exceptions as necessary. We might refactor this again later to remove
redundant code :)

* Disable response parsing when TLS interception is enabled. See issue #127

* remove unused imports

* Within webserver parse pipelined requests only if we have a route

* Add ShortLinkPlugin plugin

* Add more shortlinks

* Add ShortLinkPlugin to README.md

* Add path forwarding too instead of leaving as excercise ;)

* Add shortlink to TOC

* Ensure no socket leaks

* Ensure no leaks

* Naming

* Default number of clients 1

* Avoid shortlinking localhost

* Stress more
2019-10-15 23:56:39 -07:00
Abhinav Singh dc560be6ea
Add --timeout flag with default value of 10 second. (#129)
* Add --timeout flag with default value of 5.  This value was previously hardcoded to 30

* --timeout=10 by default

* Dispatch 408 timeout when connection is dropped due to inactivity

* Add httpStatusCodes named tuple

* Update plugin client connection reference after TLS connection upgrade
2019-10-12 21:02:17 -07:00
Abhinav Singh 908c0b5855
Chrome Devtool Integration first steps (#109)
* Initialize skeleton electron app

* Attempt to open devtools

* Electron free

* Initialize public/devtools

* Add basic support for static file serving and chrome devtools.

1. No cache header management for static file serving yet.
2. No chunked encoded responses for static files yet.
3. Chrome Devtool initialization.

* Fix static serving with query params

* profile using py-spy

* Complete websocket client loop

* lint check

* Add support for building websocket frames

* Remove redundant CDT params

* Lint check

* Refactor web server base plugin name

* Devtools integrated, need more polish

* Add START_TIME global var

* lint fix

* Remove outdated chrome rdp

* Add FAQs

* Add FAQs

* socket_connection decorator + context manager

* Defer SSL handshake and plugin initialize until protocol handler thread
has started.

This is a follow up to this PR
https://github.com/abhinavsingh/proxy.py/pull/111

* Add tests for new_socket_connection and its friend socket_connection

* Address an issue which came back after being fixed in https://github.com/abhinavsingh/proxy.py/pull/92

* Lint fixes

* uff ye str and bytes

* Remove explicit flushes outside of write ready descriptor handlers

* add links to import proxy

* Only try websocket upgrade if a route is registered

* Add plugin_examples.WebServerPlugin and use precision logging for levelname

* Remove redundant comments

* Add --devtools-ws-path flag

* Add on_websocket_open and on_websocket_close callbacks

* Add empty stubs for incomplete CDT responses

* Ensure client is ready before final flush

* Shutdown on write side of socket, may be client is still reading

* Since client.closed can be set, explicitly call client.connection.closed

* Add ModifyPostDataPlugin example.

Was first asked and referenced here
https://github.com/abhinavsingh/proxy.py/issues/115

* Start adding TestHttpProxyPlugin

* Fixes #116
2019-10-09 22:36:47 -07:00
Abhinav Singh 397f92e561
Can serve proxy traffic over HTTPS (#85)
* Can serve proxy traffic over HTTPS.

Pass --certfile and --keyfile arguments to enable SSL/TLS layer.
Example:

1) Start proxy.py as

./proxy.py --certfile cert.pem --keyfile key.pem

2) Then verify via curl

curl -v -x https://localhost:8899 --proxy-cacert cert.pem http://httpbin.org/get

* Add --ignore-missing-imports for missing multiprocessing.reduction module

* mock socket.fromfd to avoid sys calls

* Add HTTPS info

* Add TLS encryption

* Add TLS encryption
2019-09-23 17:38:10 -07:00
Abhinav Singh c3cd9be816 Add --disable-headers option. Fixes #31 2019-09-02 15:58:37 -07:00
Abhinav Singh 6b64c23c47 Prepare for v0.3 release 2019-02-08 18:57:44 -08:00
Abhinav Singh fcdae0e9c4 Fixes #20 HttpParser bug content-length is missing 2018-12-22 12:43:17 -08:00
Abhinav Singh 917564b1d2 Fix Python3 warnings thrown while running tests
1. can't concat str to bytes
2. ResourceWarning: unclosed
2018-12-10 14:43:44 +05:30
Abhinav Singh 5383abf195 PEP8 formatting 2018-10-17 14:01:19 +05:30
Abhinav Singh 755f0e07a1 variablize reply pkt, ignore, Makefile 2013-12-24 03:21:45 +05:30
Abhinav Singh 7d8a37ab34 setup.py and argparse 2013-08-20 14:56:32 +05:30
Abhinav Singh 75044a72d9 first commit 2013-08-20 03:04:30 +05:30