Rooba
/
proxy.py
mirror of https://github.com/abhinavsingh/proxy.py.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
proxy.py/helper/monitor_open_files.sh

35 lines
1.0 KiB
Bash
Raw Normal View History

Threadless execution using coroutines (#134) * Workers need not register/unregister sock for every loop * No need of explicit socket.settimeout(0) which is same as socket.setblocking(False) * Remove settimeout assertion * Only store sender side of Pipe(). Also ensure both end of the Pipe() are closed on shutdown * Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes * Use time.time throughout. Remove incomplete test_cache_responses_plugin to avoid resource leak in tests * Remove unused * Wrap selector register/unregister within a context manager * Refactor in preparation of threadless request handling * MyPy generator fix * Add --threadless flag * Internally call them acceptors * Internally use acceptors * Add Threadless class. Also no need to pass family over pipe to acceptors. * Make threadless work for a single client :) * Threadless is soon be our default * Close client queue * Use context manager for register/unregister * Fix Acceptor tests broken after refactoring * Use asyncio tasks to invoke ProtocolHandle.handle_events This gives all client threads a chance to respond without waiting for other handlers to return. * Explicitly initialize event loop per Threadless process * Mypy fixes * Add ThreadlessWork abstract class implemented by ProtocolHandler * Add benchmark.py Avoid TIME_WAIT by properly shutting down the connection. * Add benchmark.py as part of testing workflow * When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state * MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence * Unwrap if wrapped before shutdown * Unwrap if wrapped before shutdown * socket.SHUT_RDWR will cause leaks * MyPy * Add instructions for monitor.sh * Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized * Add Fast & Scalable section * Update internal classes section * Dont print out local dir path in help text :) * Refactor * Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang * Add chrome_with_proxy.sh helper script * Handle OSError during client.flush which can happen due to invalid protocol type for socket error * Remove redundant e * Add classmethods to quickly construct a parser object * Don't raise from TcpConnection abstract class. This allows both client/socket side of communication to handle exceptions as necessary. We might refactor this again later to remove redundant code :) * Disable response parsing when TLS interception is enabled. See issue #127 * remove unused imports * Within webserver parse pipelined requests only if we have a route * Add ShortLinkPlugin plugin * Add more shortlinks * Add ShortLinkPlugin to README.md * Add path forwarding too instead of leaving as excercise ;) * Add shortlink to TOC * Ensure no socket leaks * Ensure no leaks * Naming * Default number of clients 1 * Avoid shortlinking localhost * Stress more
2019-10-16 06:56:39 +00:00
#!/bin/bash
# proxy.py
# ~~~~~~~~
Test refactor + Docker image CI (#154) * Move tests into individual modules too * Ensure one test class per file * Fix docker image after refactoring * Add github actions workflow for building docker image * Fix image name * Setup python required for extracting proxy version * Version will also require deps
2019-10-30 03:41:39 +00:00
# ⚡⚡⚡ Fast, Lightweight, Programmable, TLS interception capable
# proxy server for Application debugging, testing and development.
Threadless execution using coroutines (#134) * Workers need not register/unregister sock for every loop * No need of explicit socket.settimeout(0) which is same as socket.setblocking(False) * Remove settimeout assertion * Only store sender side of Pipe(). Also ensure both end of the Pipe() are closed on shutdown * Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes * Use time.time throughout. Remove incomplete test_cache_responses_plugin to avoid resource leak in tests * Remove unused * Wrap selector register/unregister within a context manager * Refactor in preparation of threadless request handling * MyPy generator fix * Add --threadless flag * Internally call them acceptors * Internally use acceptors * Add Threadless class. Also no need to pass family over pipe to acceptors. * Make threadless work for a single client :) * Threadless is soon be our default * Close client queue * Use context manager for register/unregister * Fix Acceptor tests broken after refactoring * Use asyncio tasks to invoke ProtocolHandle.handle_events This gives all client threads a chance to respond without waiting for other handlers to return. * Explicitly initialize event loop per Threadless process * Mypy fixes * Add ThreadlessWork abstract class implemented by ProtocolHandler * Add benchmark.py Avoid TIME_WAIT by properly shutting down the connection. * Add benchmark.py as part of testing workflow * When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state * MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence * Unwrap if wrapped before shutdown * Unwrap if wrapped before shutdown * socket.SHUT_RDWR will cause leaks * MyPy * Add instructions for monitor.sh * Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized * Add Fast & Scalable section * Update internal classes section * Dont print out local dir path in help text :) * Refactor * Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang * Add chrome_with_proxy.sh helper script * Handle OSError during client.flush which can happen due to invalid protocol type for socket error * Remove redundant e * Add classmethods to quickly construct a parser object * Don't raise from TcpConnection abstract class. This allows both client/socket side of communication to handle exceptions as necessary. We might refactor this again later to remove redundant code :) * Disable response parsing when TLS interception is enabled. See issue #127 * remove unused imports * Within webserver parse pipelined requests only if we have a route * Add ShortLinkPlugin plugin * Add more shortlinks * Add ShortLinkPlugin to README.md * Add path forwarding too instead of leaving as excercise ;) * Add shortlink to TOC * Ensure no socket leaks * Ensure no leaks * Naming * Default number of clients 1 * Avoid shortlinking localhost * Stress more
2019-10-16 06:56:39 +00:00
#
# :copyright: (c) 2013-present by Abhinav Singh and contributors.
# :license: BSD, see LICENSE for more details.
#
# Usage
Update README.md (#152) * Update flags * Update debugging instructions and run instructions for develops * Update references to plugins directory * For readability add sections for run from command line using pip * Move internal doc under developer section * Add option to pass fully-qualified plugin path
2019-10-28 23:14:34 +00:00
# ./monitor_open_files <proxy-py-pid>
Threadless execution using coroutines (#134) * Workers need not register/unregister sock for every loop * No need of explicit socket.settimeout(0) which is same as socket.setblocking(False) * Remove settimeout assertion * Only store sender side of Pipe(). Also ensure both end of the Pipe() are closed on shutdown * Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes * Use time.time throughout. Remove incomplete test_cache_responses_plugin to avoid resource leak in tests * Remove unused * Wrap selector register/unregister within a context manager * Refactor in preparation of threadless request handling * MyPy generator fix * Add --threadless flag * Internally call them acceptors * Internally use acceptors * Add Threadless class. Also no need to pass family over pipe to acceptors. * Make threadless work for a single client :) * Threadless is soon be our default * Close client queue * Use context manager for register/unregister * Fix Acceptor tests broken after refactoring * Use asyncio tasks to invoke ProtocolHandle.handle_events This gives all client threads a chance to respond without waiting for other handlers to return. * Explicitly initialize event loop per Threadless process * Mypy fixes * Add ThreadlessWork abstract class implemented by ProtocolHandler * Add benchmark.py Avoid TIME_WAIT by properly shutting down the connection. * Add benchmark.py as part of testing workflow * When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state * MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence * Unwrap if wrapped before shutdown * Unwrap if wrapped before shutdown * socket.SHUT_RDWR will cause leaks * MyPy * Add instructions for monitor.sh * Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized * Add Fast & Scalable section * Update internal classes section * Dont print out local dir path in help text :) * Refactor * Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang * Add chrome_with_proxy.sh helper script * Handle OSError during client.flush which can happen due to invalid protocol type for socket error * Remove redundant e * Add classmethods to quickly construct a parser object * Don't raise from TcpConnection abstract class. This allows both client/socket side of communication to handle exceptions as necessary. We might refactor this again later to remove redundant code :) * Disable response parsing when TLS interception is enabled. See issue #127 * remove unused imports * Within webserver parse pipelined requests only if we have a route * Add ShortLinkPlugin plugin * Add more shortlinks * Add ShortLinkPlugin to README.md * Add path forwarding too instead of leaving as excercise ;) * Add shortlink to TOC * Ensure no socket leaks * Ensure no leaks * Naming * Default number of clients 1 * Avoid shortlinking localhost * Stress more
2019-10-16 06:56:39 +00:00
#
# Alternately, just run:
# watch -n 1 'lsof -i TCP:8899 | grep -v LISTEN'
PROXY_PY_PID=$1
Update README.md (#152) * Update flags * Update debugging instructions and run instructions for develops * Update references to plugins directory * For readability add sections for run from command line using pip * Move internal doc under developer section * Add option to pass fully-qualified plugin path
2019-10-28 23:14:34 +00:00
if [[ -z "$PROXY_PY_PID" ]]; then
Threadless execution using coroutines (#134) * Workers need not register/unregister sock for every loop * No need of explicit socket.settimeout(0) which is same as socket.setblocking(False) * Remove settimeout assertion * Only store sender side of Pipe(). Also ensure both end of the Pipe() are closed on shutdown * Make now global. Also we seem to be using datetime.utcnow and time.time for similar purposes * Use time.time throughout. Remove incomplete test_cache_responses_plugin to avoid resource leak in tests * Remove unused * Wrap selector register/unregister within a context manager * Refactor in preparation of threadless request handling * MyPy generator fix * Add --threadless flag * Internally call them acceptors * Internally use acceptors * Add Threadless class. Also no need to pass family over pipe to acceptors. * Make threadless work for a single client :) * Threadless is soon be our default * Close client queue * Use context manager for register/unregister * Fix Acceptor tests broken after refactoring * Use asyncio tasks to invoke ProtocolHandle.handle_events This gives all client threads a chance to respond without waiting for other handlers to return. * Explicitly initialize event loop per Threadless process * Mypy fixes * Add ThreadlessWork abstract class implemented by ProtocolHandler * Add benchmark.py Avoid TIME_WAIT by properly shutting down the connection. * Add benchmark.py as part of testing workflow * When e2e encryption is enabled, unwrap socket before shutdown to ensure CLOSED state * MyPy fixes, Union should have worked, but likely unwrap is not part of socket.socket hence * Unwrap if wrapped before shutdown * Unwrap if wrapped before shutdown * socket.SHUT_RDWR will cause leaks * MyPy * Add instructions for monitor.sh * Avoid recursive exception in new_socket_connection and only invoke plugins/shutdown if server connection was initialized * Add Fast & Scalable section * Update internal classes section * Dont print out local dir path in help text :) * Refactor * Fix a bug where response parser for HTTP only requests was reused for pipelined requests resulting in a hang * Add chrome_with_proxy.sh helper script * Handle OSError during client.flush which can happen due to invalid protocol type for socket error * Remove redundant e * Add classmethods to quickly construct a parser object * Don't raise from TcpConnection abstract class. This allows both client/socket side of communication to handle exceptions as necessary. We might refactor this again later to remove redundant code :) * Disable response parsing when TLS interception is enabled. See issue #127 * remove unused imports * Within webserver parse pipelined requests only if we have a route * Add ShortLinkPlugin plugin * Add more shortlinks * Add ShortLinkPlugin to README.md * Add path forwarding too instead of leaving as excercise ;) * Add shortlink to TOC * Ensure no socket leaks * Ensure no leaks * Naming * Default number of clients 1 * Avoid shortlinking localhost * Stress more
2019-10-16 06:56:39 +00:00
echo "PROXY_PY_PID required as argument."
exit 1
fi
OPEN_FILES_BY_MAIN=$(lsof -p "$PROXY_PY_PID" | wc -l)
echo "[$PROXY_PY_PID] Main process: $OPEN_FILES_BY_MAIN"
pgrep -P "$PROXY_PY_PID" | while read -r acceptorPid; do
OPEN_FILES_BY_ACCEPTOR=$(lsof -p "$acceptorPid" | wc -l)
echo "[$acceptorPid] Acceptor process: $OPEN_FILES_BY_ACCEPTOR"
pgrep -P "$acceptorPid" | while read -r threadlessPid; do
OPEN_FILES_BY_THREADLESS=$(lsof -p "$threadlessPid" | wc -l)
echo " [$threadlessPid] Threadless process: $OPEN_FILES_BY_THREADLESS"
done
done