* Introduce proxy.Proxy context manager.
This is similar to already existing context manager `start` but
`proxy.Proxy` is a class with __enter__ and __exit__ methods. This
allows usage of `proxy.Proxy` both as context manager and for manually
setup and teardown of `proxy.py` during test setUpClass and
teardownClass methods.
* Gracefully shutdown threadless processes
* Update tests and add a VCR method. See #184
* Refactor routes
* Add Proxy to __all__
* Move TestCase under proxy.testing and test_embed.py under tests.embed module to avoid conflict with http module due to a http directory under proxy folder
* Add a base cache plugin class which can be customized for custom cache behaviors
* See #184. Add VCRPlugin which can be enabled within tests using a context manager, e.g. with self.vcr(): ...
* Make cache plugin pluggable + make cache storage pluggable
* Make dashboard npm module agnostic of top level directory
* Symlink dashboard public folder
* Dump devtools within dashboard public folder
* Remove unused 3rd party js
* Update doc and banner
* Update banner to match GitHub
* Update older banners too
* Add update_desc to .gitignore
* Update banner for dashboard to match github
* also update html, js, css
* Update dev guide
* Move plugin_examples/ as proxy.plugin
* Update proxy.plugin ref path in readme
* Remove unnecessary port flag
* Remove plugin_examples from github workflows
* dashboard folder is a npm package not python package anymore
* Plugins can now be tried using Docker image
* Allow resources to load from http and ws when running w/o https
* Move dashboard backend (dashboard.py) within proxy module. Now shipped with pip install proxy.py
* Update ref to dashboard backend in github workflows
* Add git-pre-commit hook file.
Enable it by symlinking as .git/hooks/pre-commit
* Also enable static server for dashboard serving
* Refine docs
* Decouple relay from dashboard.
Will be re-used by devtools protocol plugin.
* Just have a single manager for all eventing
* Ofcourse managers cant be shared across processes
* Remove unused
* Add DevtoolsProtocolPlugin
* Emit REQUEST_COMPLETE core event
* Emit only if --enable-events used
* Add event emitter for response cycle
* Fill up core events to devtools protocol expectations
* Serve static content with Cache-Control header and gzip compression
* Add PWA manifest.json and icons from sample PWA apps (replace later)
* Catch any exception and be ssl agnostic
* Add CSP headers and avoid inline scripts
* Re-enable iframe and deobfuscation
* Embed plugins within <section/> block
* Make tab switching agnostic of block name
* Add support for browser history on tab change
* Default hash to #home
* Switch to tab if hash is already set
* Expand canvas to fill screen even without content
* Remove inline css for embedded devtools
* Make dashboard backend websocket API pluggable
* doc
* Dynamically load devtools instead of on page load
* Add support for passing flags as kwargs to main / start methods.
* Fix tests for refactored code
* Allow proxy.main, proxy.start, proxy.TestCase.
Also update README.md to reflect the same.
* Use Any for **opts
* Move main as __init__ to avoid name conflicts
* Fix tests
* Update setup.py entry_point
* Explicitly install requirements before setup.py
* Explicitly mention packages of interest
* ipv6 fails on ubuntu, use ipv4
* Make typing-extensions optional
* Instead of putting it all under __init__.py, move main.py to proxy.py
* Simply make setup.py module free
* autopep8
* Explicitly link version changelog in TOC
* Separate out app header body builder
* Ensure unsubscribe when disabling inspection. Fixes#164
* Avoid creation of new manager per dashboard instance.
* Add UI header for all plugins (tabs)
* Ensure app body for all plugin skeleton
* Move app-header and app-body within core for consistent dashboard look and feel
* Consistent UI header body for plugins
* autopep8
* make devtools
* convert to es6
* Add inspect_traffic plugin devtools app
* trigger re-build, github UI is stuck
* Dynamically load devtools within inspect traffic view
* Just copy devtools into public/dashboard folder
* Works but not how we wanted, devtools takes over entire body and doesnt contain itself within a div
* Explicitly link version changelog in TOC
* Separate out app header body builder
* Ensure unsubscribe when disabling inspection. Fixes#164
* Avoid creation of new manager per dashboard instance.
* Add UI header for all plugins (tabs)
* Ensure app body for all plugin skeleton
* Move app-header and app-body within core for consistent dashboard look and feel
* Consistent UI header body for plugins
* autopep8
* Add demonstration of how to use proxy.py within Python application unittests
* mypy fixes
* test_with_proxy example
* Add docs for proxy.main.TestCase.
Also wait for proxy.py server to come up before running the tests.
* Introduce sendMessage websocket api which allows for callbacks, deprecate lastPingId in favor of callbacks
* Let InspectTrafficPlugin handle all pushed inspection events
* Separate out files for different responsibilities.
1. Add src/plugins directory. This directory holds one typescript file
per plugin. Each plugin is optionally can be displayed as a tab on
the UI.
2. Move WebsocketApi to ws.ts. This file contains all websocket APIs
provided by dashboard.py backend.
* Make dashboard pluggable
* Move devtools under core too
* Register tabs dynamically
* Typescript fixes for abstract interfaces
* Initialize plugin app body skeleton
* Call activated / deactivated on tab change
* Move plugin name within plugin classes and initialize plugin within proxy dashboard constructor
* templatize api development plugin
* eslint fixes
* use globs
* Remove useless constructors
* Add utilities to generate private key and public keys with alternate cnames
* Add separate package proxy.py-plugins, fixes#156
* Generate certificates to comply with Mac requirements.
* Add utility for CSR generation and signing
* Fixes#161
* Add initial pki tests
* Refactor Makefile and add dashboard setup.py
* Package dashboard as proxy.py-dashboard pip package
* Give dashboard releases its own version
* Fix lib-package reference
* Move tests into individual modules too
* Ensure one test class per file
* Fix docker image after refactoring
* Add github actions workflow for building docker image
* Fix image name
* Setup python required for extracting proxy version
* Version will also require deps
* Update flags
* Update debugging instructions and run instructions for develops
* Update references to plugins directory
* For readability add sections for run from command line using pip
* Move internal doc under developer section
* Add option to pass fully-qualified plugin path
* Remove redundant variables
* Initialize frontend dashboard app (written in typescript)
* Add a WebsocketFrame.text method to quickly build a text frame raw packet, also close connection for static file serving, atleast Google Chrome seems to hang up instead of closing the connection
* Add read_and_build_static_file_response method for reusability in plugins
* teardown websocket connection when opcode CONNECTION_CLOSE is received
* First draft of proxy.py dashboard
* Remove uglify, obfuscator is superb enough
* Correct generic V
* First draft of dashboard
* ProtocolConfig is now Flags
* First big refactor toward no-single-file-module
* Working tests
* Update dashboard for refactored imports
* Remove proxy.py as now we can just call python -m proxy -h
* Fix setup.py for refactored code
* Banner update
* Lint check
* Fix dashboard static serving and no UNDER_TEST constant necessary
* Add support for plugin imports when specified in path/to/module.MyPlugin
* Update README with instructions to run proxy.py after refactor
* Move dashboard under /dashboard path
* Rename to devtools.ts
* remove unused
* Update github workflow for new directory structure
* Update test command too
* Fix coverage generation
* *.py is an invalid syntax on windows
* No * on windows
* Enable execution via github zip downloads
* Github Zip downloads cannot be executed as Github puts project under a folder named after Github project, this breaks python interpreter expectation of finding a __main__.py in the root directory
* Forget zip runs for now
* Initialize ProxyDashboard on page load rather than within typescript i.e. on script load
* Enforce eslint with standard style
* Add .editorconfig to make editor compatible with various style requirements (Makefile, Typescript, Python)
* Remove extra empty line
* Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected
* Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected
* Fix tests
* Move common code under common sub-module
* Move flags under common module
* Move acceptor under core
* Move connection under core submodule
* Move chunk_parser under http
* Move http_parser as http/parser
* Move http_methods as http/methods
* Move http_proxy as http/proxy
* Move web_server as http/server
* Move status_codes as http/codes
* move websocket as http/websocket
* Move exception under http/exception, also move http/proxy exceptions under http/exceptions
* move protocol_handler as http/handler
* move devtools as http/devtools
* Move version under common/version
* Lifecycle if now core Event
* autopep8
* Add core event queue
* Register / unregister handler
* Enable inspection support for frontend dashboard
* Dont give an illusion of exception for HttpProtocolExceptions
* Update readme for refactored codebase
* DictQueueType everywhere
* Move all websocket API related code under WebsocketApi class
* Inspection enabled on tab switch.
1. Additionally now acceptors are assigned an int id.
2. Fix tests to match change in constructor.
* Corresponding ends of the work queues can be closed immediately.
Since work queues between AcceptorPool and Acceptor process is used only
once, close corresponding ends asap instead of at shutdown.
* No need of a manager for shared multiprocess Lock.
This unnecessarily creates additional manager process.
* Move threadless into its own module
* Merge acceptor and acceptor_pool tests
* Defer os.close
* Change content display with tab clicks.
Also ensure relay manager shutdown.
* Remove --cov flags
* Use right type for SyncManager
* Ensure coverage again
* Print help to discover flags, --cov certainly not available on Travis for some reason
* Add pytest-cov to requirements-testing
* Re-add windows on .travis also add changelog to readme
* Use 3.7 and no pip upgrade since it fails on travis windows
* Attempt to fix pip install on windows
* Disable windows on travis, it fails and uses 3.8. Try reporting coverage from github actions
* Move away from coveralls, use codecov
* Codecov app installation either didnt work or token still needs to be passed
* Remove travis CI
* Use https://github.com/codecov/codecov-action for coverage uploads
* Remove run codecov
* Ha, codecov action only works on linux, what a mess
* Add cookie.js though unable to use it with es5/es6 modules yet
* Enable testing for python 3.8 also Build dashboard during testing
* No python 3.8 on github actions yet
* Autopep8
* Add separate workflows for library (python) and dashboard (node) app
* Type jobs not job
* Add checkout
* Fix parsing node version
* Fix dashboard build on windows
* Show codecov instead of coveralls
* Add pipeline response parsing tests
* build_http_response now only adds content-length if transfer-encoding is
not provided.
Also return pending raw chunks from ChunkParser so that we can parse
pipelined chunk responses.
* Workers need not register/unregister sock for every loop
* No need of explicit socket.settimeout(0) which is same as socket.setblocking(False)
* Remove settimeout assertion
* Only store sender side of Pipe(). Also ensure both end of the Pipe() are closed on shutdown
* Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes
* Use time.time throughout. Remove incomplete test_cache_responses_plugin to avoid resource leak in tests
* Remove unused
* Wrap selector register/unregister within a context manager
* Refactor in preparation of threadless request handling
* MyPy generator fix
* Add --threadless flag
* Internally call them acceptors
* Internally use acceptors
* Add Threadless class.
Also no need to pass family over pipe to acceptors.
* Make threadless work for a single client :)
* Threadless is soon be our default
* Close client queue
* Use context manager for register/unregister
* Fix Acceptor tests broken after refactoring
* Use asyncio tasks to invoke ProtocolHandle.handle_events
This gives all client threads a chance to respond without
waiting for other handlers to return.
* Explicitly initialize event loop per Threadless process
* Mypy fixes
* Add ThreadlessWork abstract class implemented by ProtocolHandler
* Add benchmark.py
Avoid TIME_WAIT by properly shutting down the connection.
* Add benchmark.py as part of testing workflow
* When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state
* MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence
* Unwrap if wrapped before shutdown
* Unwrap if wrapped before shutdown
* socket.SHUT_RDWR will cause leaks
* MyPy
* Add instructions for monitor.sh
* Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized
* Add Fast & Scalable section
* Update internal classes section
* Dont print out local dir path in help text :)
* Refactor
* Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang
* Add chrome_with_proxy.sh helper script
* Handle OSError during client.flush which can happen due to invalid protocol type for socket error
* Remove redundant e
* Add classmethods to quickly construct a parser object
* Don't raise from TcpConnection abstract class.
This allows both client/socket side of communication to handle
exceptions as necessary. We might refactor this again later to remove
redundant code :)
* Disable response parsing when TLS interception is enabled. See issue #127
* remove unused imports
* Within webserver parse pipelined requests only if we have a route
* Add ShortLinkPlugin plugin
* Add more shortlinks
* Add ShortLinkPlugin to README.md
* Add path forwarding too instead of leaving as excercise ;)
* Add shortlink to TOC
* Ensure no socket leaks
* Ensure no leaks
* Naming
* Default number of clients 1
* Avoid shortlinking localhost
* Stress more
* Add tests for plugin_examples.* to ensure we never break functionality
* Add tests for plugin_examples.*
* Test man in the middle
* Lint fixes
* Checkin
* Add tests for plugin examples with TLS encryption enabled
* Add --timeout flag with default value of 5. This value was previously hardcoded to 30
* --timeout=10 by default
* Dispatch 408 timeout when connection is dropped due to inactivity
* Add httpStatusCodes named tuple
* Update plugin client connection reference after TLS connection upgrade
* Add tests for is_http_1_1_keep_alive
* Add ModifyPostDataPlugin in README
* Fixes#126
* Refactor HttpProxyBasePlugin API
* before_upstream_connection too can drop request by returning None
* Remove HTTP Server startup during tests, no longer used
* Removed unused imports
* Simplify load_plugins
* Add tests to verify certificate generation
* Separate out tests for ProtocolHandler and WebServerPlugin
* Keep-alive connections for web server.
TODO: Only keep-alivei if HTTP/1.1
* Add request.path to avoid build_url repeatedly whose name is also slightly misleading
* Fix example usage of request.path
* Pipeline only for HTTP/1.1
* Lint fix
* Teardown HTTP/1.1 keep-alive request when Connection: close header is sent
* Add instructions on how to build docker image locally
* Move access_log to separate function for pretty logging
Abhinav Singh