Rooba
/
proxy.py
mirror of https://github.com/abhinavsingh/proxy.py.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
proxy.py/proxy/__init__.py

26 lines
822 B
Python
Raw Normal View History

Proxy.py Dashboard (#141) * Remove redundant variables * Initialize frontend dashboard app (written in typescript) * Add a WebsocketFrame.text method to quickly build a text frame raw packet, also close connection for static file serving, atleast Google Chrome seems to hang up instead of closing the connection * Add read_and_build_static_file_response method for reusability in plugins * teardown websocket connection when opcode CONNECTION_CLOSE is received * First draft of proxy.py dashboard * Remove uglify, obfuscator is superb enough * Correct generic V * First draft of dashboard * ProtocolConfig is now Flags * First big refactor toward no-single-file-module * Working tests * Update dashboard for refactored imports * Remove proxy.py as now we can just call python -m proxy -h * Fix setup.py for refactored code * Banner update * Lint check * Fix dashboard static serving and no UNDER_TEST constant necessary * Add support for plugin imports when specified in path/to/module.MyPlugin * Update README with instructions to run proxy.py after refactor * Move dashboard under /dashboard path * Rename to devtools.ts * remove unused * Update github workflow for new directory structure * Update test command too * Fix coverage generation * *.py is an invalid syntax on windows * No * on windows * Enable execution via github zip downloads * Github Zip downloads cannot be executed as Github puts project under a folder named after Github project, this breaks python interpreter expectation of finding a __main__.py in the root directory * Forget zip runs for now * Initialize ProxyDashboard on page load rather than within typescript i.e. on script load * Enforce eslint with standard style * Add .editorconfig to make editor compatible with various style requirements (Makefile, Typescript, Python) * Remove extra empty line * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Fix tests * Move common code under common sub-module * Move flags under common module * Move acceptor under core * Move connection under core submodule * Move chunk_parser under http * Move http_parser as http/parser * Move http_methods as http/methods * Move http_proxy as http/proxy * Move web_server as http/server * Move status_codes as http/codes * move websocket as http/websocket * Move exception under http/exception, also move http/proxy exceptions under http/exceptions * move protocol_handler as http/handler * move devtools as http/devtools * Move version under common/version * Lifecycle if now core Event * autopep8 * Add core event queue * Register / unregister handler * Enable inspection support for frontend dashboard * Dont give an illusion of exception for HttpProtocolExceptions * Update readme for refactored codebase * DictQueueType everywhere * Move all websocket API related code under WebsocketApi class * Inspection enabled on tab switch. 1. Additionally now acceptors are assigned an int id. 2. Fix tests to match change in constructor. * Corresponding ends of the work queues can be closed immediately. Since work queues between AcceptorPool and Acceptor process is used only once, close corresponding ends asap instead of at shutdown. * No need of a manager for shared multiprocess Lock. This unnecessarily creates additional manager process. * Move threadless into its own module * Merge acceptor and acceptor_pool tests * Defer os.close * Change content display with tab clicks. Also ensure relay manager shutdown. * Remove --cov flags * Use right type for SyncManager * Ensure coverage again * Print help to discover flags, --cov certainly not available on Travis for some reason * Add pytest-cov to requirements-testing * Re-add windows on .travis also add changelog to readme * Use 3.7 and no pip upgrade since it fails on travis windows * Attempt to fix pip install on windows * Disable windows on travis, it fails and uses 3.8. Try reporting coverage from github actions * Move away from coveralls, use codecov * Codecov app installation either didnt work or token still needs to be passed * Remove travis CI * Use https://github.com/codecov/codecov-action for coverage uploads * Remove run codecov * Ha, codecov action only works on linux, what a mess * Add cookie.js though unable to use it with es5/es6 modules yet * Enable testing for python 3.8 also Build dashboard during testing * No python 3.8 on github actions yet * Autopep8 * Add separate workflows for library (python) and dashboard (node) app * Type jobs not job * Add checkout * Fix parsing node version * Fix dashboard build on windows * Show codecov instead of coveralls
2019-10-28 21:57:33 +00:00
# -*- coding: utf-8 -*-
"""
proxy.py
~~~~~~~~
Doc & Banner update to match GitHub (#182) * Update doc and banner * Update banner to match GitHub * Update older banners too * Add update_desc to .gitignore * Update banner for dashboard to match github * also update html, js, css
2019-11-19 04:45:51 +00:00
⚡⚡⚡ Fast, Lightweight, Pluggable, TLS interception capable proxy server focused on
Network monitoring, controls & Application development, testing, debugging.
Proxy.py Dashboard (#141) * Remove redundant variables * Initialize frontend dashboard app (written in typescript) * Add a WebsocketFrame.text method to quickly build a text frame raw packet, also close connection for static file serving, atleast Google Chrome seems to hang up instead of closing the connection * Add read_and_build_static_file_response method for reusability in plugins * teardown websocket connection when opcode CONNECTION_CLOSE is received * First draft of proxy.py dashboard * Remove uglify, obfuscator is superb enough * Correct generic V * First draft of dashboard * ProtocolConfig is now Flags * First big refactor toward no-single-file-module * Working tests * Update dashboard for refactored imports * Remove proxy.py as now we can just call python -m proxy -h * Fix setup.py for refactored code * Banner update * Lint check * Fix dashboard static serving and no UNDER_TEST constant necessary * Add support for plugin imports when specified in path/to/module.MyPlugin * Update README with instructions to run proxy.py after refactor * Move dashboard under /dashboard path * Rename to devtools.ts * remove unused * Update github workflow for new directory structure * Update test command too * Fix coverage generation * *.py is an invalid syntax on windows * No * on windows * Enable execution via github zip downloads * Github Zip downloads cannot be executed as Github puts project under a folder named after Github project, this breaks python interpreter expectation of finding a __main__.py in the root directory * Forget zip runs for now * Initialize ProxyDashboard on page load rather than within typescript i.e. on script load * Enforce eslint with standard style * Add .editorconfig to make editor compatible with various style requirements (Makefile, Typescript, Python) * Remove extra empty line * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Fix tests * Move common code under common sub-module * Move flags under common module * Move acceptor under core * Move connection under core submodule * Move chunk_parser under http * Move http_parser as http/parser * Move http_methods as http/methods * Move http_proxy as http/proxy * Move web_server as http/server * Move status_codes as http/codes * move websocket as http/websocket * Move exception under http/exception, also move http/proxy exceptions under http/exceptions * move protocol_handler as http/handler * move devtools as http/devtools * Move version under common/version * Lifecycle if now core Event * autopep8 * Add core event queue * Register / unregister handler * Enable inspection support for frontend dashboard * Dont give an illusion of exception for HttpProtocolExceptions * Update readme for refactored codebase * DictQueueType everywhere * Move all websocket API related code under WebsocketApi class * Inspection enabled on tab switch. 1. Additionally now acceptors are assigned an int id. 2. Fix tests to match change in constructor. * Corresponding ends of the work queues can be closed immediately. Since work queues between AcceptorPool and Acceptor process is used only once, close corresponding ends asap instead of at shutdown. * No need of a manager for shared multiprocess Lock. This unnecessarily creates additional manager process. * Move threadless into its own module * Merge acceptor and acceptor_pool tests * Defer os.close * Change content display with tab clicks. Also ensure relay manager shutdown. * Remove --cov flags * Use right type for SyncManager * Ensure coverage again * Print help to discover flags, --cov certainly not available on Travis for some reason * Add pytest-cov to requirements-testing * Re-add windows on .travis also add changelog to readme * Use 3.7 and no pip upgrade since it fails on travis windows * Attempt to fix pip install on windows * Disable windows on travis, it fails and uses 3.8. Try reporting coverage from github actions * Move away from coveralls, use codecov * Codecov app installation either didnt work or token still needs to be passed * Remove travis CI * Use https://github.com/codecov/codecov-action for coverage uploads * Remove run codecov * Ha, codecov action only works on linux, what a mess * Add cookie.js though unable to use it with es5/es6 modules yet * Enable testing for python 3.8 also Build dashboard during testing * No python 3.8 on github actions yet * Autopep8 * Add separate workflows for library (python) and dashboard (node) app * Type jobs not job * Add checkout * Fix parsing node version * Fix dashboard build on windows * Show codecov instead of coveralls
2019-10-28 21:57:33 +00:00
:copyright: (c) 2013-present by Abhinav Singh and contributors.
:license: BSD, see LICENSE for more details.
"""
Update outdated sections of the codebase (#670) * Remove autopep8, is redundant now after recent CI changes * Add pyenv .python-version to .gitignore * Update year * Add lib-pytest target so that pytest can run in isolation * Add git-push hook which will also run the lint. By default now git-pre-commit hook will only run pytest. * Update outdated sections of README * Update requirement to match setup.cfg install_requires * Deprecate proxy.start and TestCase.PROXY_PORT Proxy port during test is now available as self.PROXY.pool.flags.port. Also now TestCase utilize ephemeral port strategy instead of calling get_available_port utility method. * Rename to git-pre-push * Ideally public repo dont require CODECOV_TOKEN but codecov integration is broken since introduction of codecov-action@v2 (instead of codecov binary invocation) * Issue is possibly with codecov@v2 action, fallback to codecov. See https://github.com/abhinavsingh/proxy.py/runs/4110423084\?check_suite_focus\=true and https://github.com/codecov/uploader/issues/223 * Revert back to v2
2021-11-04 21:41:22 +00:00
from .proxy import entry_point, main, Proxy
Fix `--enable-dashboard` flags (#707) * Expose within __all__ * Enable `--numprocesses=auto` for `pytest.ini` * make lib-lint * Also consider `--plugins` flag when bootstrapping plugins * Add `from .dashboard import ProxyDashboard` in top-level `__init__.py` to make `ProxyDashboard` flags auto discoverable * Move `--enable-dashboard` to top-level * Move logging utility within `Logger` class * Consider comma separated --plugin and --plugins during discover_plugins * Refactor plugin related utilities in Plugins module * mypy and lint * Fix unused import * Safe to use tempdir on Github actions to avoid race conditions??? * pki (generically disk based file) based tests are flaky on macOS under parallel execution
2021-11-08 16:12:43 +00:00
from .testing import TestCase
Allow to pass flags as kwargs too in embed mode (#172) * Dynamically load devtools instead of on page load * Add support for passing flags as kwargs to main / start methods. * Fix tests for refactored code * Allow proxy.main, proxy.start, proxy.TestCase. Also update README.md to reflect the same. * Use Any for **opts * Move main as __init__ to avoid name conflicts * Fix tests * Update setup.py entry_point * Explicitly install requirements before setup.py * Explicitly mention packages of interest * ipv6 fails on ubuntu, use ipv4 * Make typing-extensions optional * Instead of putting it all under __init__.py, move main.py to proxy.py * Simply make setup.py module free * autopep8
2019-11-14 00:38:59 +00:00
__all__ = [
Devtools Protocol (#174) * Refine docs * Decouple relay from dashboard. Will be re-used by devtools protocol plugin. * Just have a single manager for all eventing * Ofcourse managers cant be shared across processes * Remove unused * Add DevtoolsProtocolPlugin * Emit REQUEST_COMPLETE core event * Emit only if --enable-events used * Add event emitter for response cycle * Fill up core events to devtools protocol expectations * Serve static content with Cache-Control header and gzip compression * Add PWA manifest.json and icons from sample PWA apps (replace later) * Catch any exception and be ssl agnostic * Add CSP headers and avoid inline scripts * Re-enable iframe and deobfuscation * Embed plugins within <section/> block * Make tab switching agnostic of block name * Add support for browser history on tab change * Default hash to #home * Switch to tab if hash is already set * Expand canvas to fill screen even without content * Remove inline css for embedded devtools * Make dashboard backend websocket API pluggable * doc
2019-11-15 03:00:07 +00:00
# PyPi package entry_point. See
Allow to pass flags as kwargs too in embed mode (#172) * Dynamically load devtools instead of on page load * Add support for passing flags as kwargs to main / start methods. * Fix tests for refactored code * Allow proxy.main, proxy.start, proxy.TestCase. Also update README.md to reflect the same. * Use Any for **opts * Move main as __init__ to avoid name conflicts * Fix tests * Update setup.py entry_point * Explicitly install requirements before setup.py * Explicitly mention packages of interest * ipv6 fails on ubuntu, use ipv4 * Make typing-extensions optional * Instead of putting it all under __init__.py, move main.py to proxy.py * Simply make setup.py module free * autopep8
2019-11-14 00:38:59 +00:00
# https://github.com/abhinavsingh/proxy.py#from-command-line-when-installed-using-pip
'entry_point',
Devtools Protocol (#174) * Refine docs * Decouple relay from dashboard. Will be re-used by devtools protocol plugin. * Just have a single manager for all eventing * Ofcourse managers cant be shared across processes * Remove unused * Add DevtoolsProtocolPlugin * Emit REQUEST_COMPLETE core event * Emit only if --enable-events used * Add event emitter for response cycle * Fill up core events to devtools protocol expectations * Serve static content with Cache-Control header and gzip compression * Add PWA manifest.json and icons from sample PWA apps (replace later) * Catch any exception and be ssl agnostic * Add CSP headers and avoid inline scripts * Re-enable iframe and deobfuscation * Embed plugins within <section/> block * Make tab switching agnostic of block name * Add support for browser history on tab change * Default hash to #home * Switch to tab if hash is already set * Expand canvas to fill screen even without content * Remove inline css for embedded devtools * Make dashboard backend websocket API pluggable * doc
2019-11-15 03:00:07 +00:00
# Embed proxy.py. See
# https://github.com/abhinavsingh/proxy.py#embed-proxypy
Update outdated sections of the codebase (#670) * Remove autopep8, is redundant now after recent CI changes * Add pyenv .python-version to .gitignore * Update year * Add lib-pytest target so that pytest can run in isolation * Add git-push hook which will also run the lint. By default now git-pre-commit hook will only run pytest. * Update outdated sections of README * Update requirement to match setup.cfg install_requires * Deprecate proxy.start and TestCase.PROXY_PORT Proxy port during test is now available as self.PROXY.pool.flags.port. Also now TestCase utilize ephemeral port strategy instead of calling get_available_port utility method. * Rename to git-pre-push * Ideally public repo dont require CODECOV_TOKEN but codecov integration is broken since introduction of codecov-action@v2 (instead of codecov binary invocation) * Issue is possibly with codecov@v2 action, fallback to codecov. See https://github.com/abhinavsingh/proxy.py/runs/4110423084\?check_suite_focus\=true and https://github.com/codecov/uploader/issues/223 * Revert back to v2
2021-11-04 21:41:22 +00:00
'main',
Devtools Protocol (#174) * Refine docs * Decouple relay from dashboard. Will be re-used by devtools protocol plugin. * Just have a single manager for all eventing * Ofcourse managers cant be shared across processes * Remove unused * Add DevtoolsProtocolPlugin * Emit REQUEST_COMPLETE core event * Emit only if --enable-events used * Add event emitter for response cycle * Fill up core events to devtools protocol expectations * Serve static content with Cache-Control header and gzip compression * Add PWA manifest.json and icons from sample PWA apps (replace later) * Catch any exception and be ssl agnostic * Add CSP headers and avoid inline scripts * Re-enable iframe and deobfuscation * Embed plugins within <section/> block * Make tab switching agnostic of block name * Add support for browser history on tab change * Default hash to #home * Switch to tab if hash is already set * Expand canvas to fill screen even without content * Remove inline css for embedded devtools * Make dashboard backend websocket API pluggable * doc
2019-11-15 03:00:07 +00:00
# Unit testing with proxy.py. See
Allow to pass flags as kwargs too in embed mode (#172) * Dynamically load devtools instead of on page load * Add support for passing flags as kwargs to main / start methods. * Fix tests for refactored code * Allow proxy.main, proxy.start, proxy.TestCase. Also update README.md to reflect the same. * Use Any for **opts * Move main as __init__ to avoid name conflicts * Fix tests * Update setup.py entry_point * Explicitly install requirements before setup.py * Explicitly mention packages of interest * ipv6 fails on ubuntu, use ipv4 * Make typing-extensions optional * Instead of putting it all under __init__.py, move main.py to proxy.py * Simply make setup.py module free * autopep8
2019-11-14 00:38:59 +00:00
# https://github.com/abhinavsingh/proxy.py#unit-testing-with-proxypy
Testing support improvements (#185) * Introduce proxy.Proxy context manager. This is similar to already existing context manager `start` but `proxy.Proxy` is a class with __enter__ and __exit__ methods. This allows usage of `proxy.Proxy` both as context manager and for manually setup and teardown of `proxy.py` during test setUpClass and teardownClass methods. * Gracefully shutdown threadless processes * Update tests and add a VCR method. See #184 * Refactor routes * Add Proxy to __all__ * Move TestCase under proxy.testing and test_embed.py under tests.embed module to avoid conflict with http module due to a http directory under proxy folder * Add a base cache plugin class which can be customized for custom cache behaviors * See #184. Add VCRPlugin which can be enabled within tests using a context manager, e.g. with self.vcr(): ... * Make cache plugin pluggable + make cache storage pluggable * Make dashboard npm module agnostic of top level directory * Symlink dashboard public folder * Dump devtools within dashboard public folder * Remove unused 3rd party js
2019-11-22 05:16:01 +00:00
'TestCase',
'Proxy',
Allow to pass flags as kwargs too in embed mode (#172) * Dynamically load devtools instead of on page load * Add support for passing flags as kwargs to main / start methods. * Fix tests for refactored code * Allow proxy.main, proxy.start, proxy.TestCase. Also update README.md to reflect the same. * Use Any for **opts * Move main as __init__ to avoid name conflicts * Fix tests * Update setup.py entry_point * Explicitly install requirements before setup.py * Explicitly mention packages of interest * ipv6 fails on ubuntu, use ipv4 * Make typing-extensions optional * Instead of putting it all under __init__.py, move main.py to proxy.py * Simply make setup.py module free * autopep8
2019-11-14 00:38:59 +00:00
]