proxy.py/proxy/core/connection.py

130 lines
3.9 KiB
Python
Raw Normal View History

Proxy.py Dashboard (#141) * Remove redundant variables * Initialize frontend dashboard app (written in typescript) * Add a WebsocketFrame.text method to quickly build a text frame raw packet, also close connection for static file serving, atleast Google Chrome seems to hang up instead of closing the connection * Add read_and_build_static_file_response method for reusability in plugins * teardown websocket connection when opcode CONNECTION_CLOSE is received * First draft of proxy.py dashboard * Remove uglify, obfuscator is superb enough * Correct generic V * First draft of dashboard * ProtocolConfig is now Flags * First big refactor toward no-single-file-module * Working tests * Update dashboard for refactored imports * Remove proxy.py as now we can just call python -m proxy -h * Fix setup.py for refactored code * Banner update * Lint check * Fix dashboard static serving and no UNDER_TEST constant necessary * Add support for plugin imports when specified in path/to/module.MyPlugin * Update README with instructions to run proxy.py after refactor * Move dashboard under /dashboard path * Rename to devtools.ts * remove unused * Update github workflow for new directory structure * Update test command too * Fix coverage generation * *.py is an invalid syntax on windows * No * on windows * Enable execution via github zip downloads * Github Zip downloads cannot be executed as Github puts project under a folder named after Github project, this breaks python interpreter expectation of finding a __main__.py in the root directory * Forget zip runs for now * Initialize ProxyDashboard on page load rather than within typescript i.e. on script load * Enforce eslint with standard style * Add .editorconfig to make editor compatible with various style requirements (Makefile, Typescript, Python) * Remove extra empty line * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Add ability to pass headers with HttpRequestRejected exception, also remove proxy agent header for HttpRequestRejected * Fix tests * Move common code under common sub-module * Move flags under common module * Move acceptor under core * Move connection under core submodule * Move chunk_parser under http * Move http_parser as http/parser * Move http_methods as http/methods * Move http_proxy as http/proxy * Move web_server as http/server * Move status_codes as http/codes * move websocket as http/websocket * Move exception under http/exception, also move http/proxy exceptions under http/exceptions * move protocol_handler as http/handler * move devtools as http/devtools * Move version under common/version * Lifecycle if now core Event * autopep8 * Add core event queue * Register / unregister handler * Enable inspection support for frontend dashboard * Dont give an illusion of exception for HttpProtocolExceptions * Update readme for refactored codebase * DictQueueType everywhere * Move all websocket API related code under WebsocketApi class * Inspection enabled on tab switch. 1. Additionally now acceptors are assigned an int id. 2. Fix tests to match change in constructor. * Corresponding ends of the work queues can be closed immediately. Since work queues between AcceptorPool and Acceptor process is used only once, close corresponding ends asap instead of at shutdown. * No need of a manager for shared multiprocess Lock. This unnecessarily creates additional manager process. * Move threadless into its own module * Merge acceptor and acceptor_pool tests * Defer os.close * Change content display with tab clicks. Also ensure relay manager shutdown. * Remove --cov flags * Use right type for SyncManager * Ensure coverage again * Print help to discover flags, --cov certainly not available on Travis for some reason * Add pytest-cov to requirements-testing * Re-add windows on .travis also add changelog to readme * Use 3.7 and no pip upgrade since it fails on travis windows * Attempt to fix pip install on windows * Disable windows on travis, it fails and uses 3.8. Try reporting coverage from github actions * Move away from coveralls, use codecov * Codecov app installation either didnt work or token still needs to be passed * Remove travis CI * Use https://github.com/codecov/codecov-action for coverage uploads * Remove run codecov * Ha, codecov action only works on linux, what a mess * Add cookie.js though unable to use it with es5/es6 modules yet * Enable testing for python 3.8 also Build dashboard during testing * No python 3.8 on github actions yet * Autopep8 * Add separate workflows for library (python) and dashboard (node) app * Type jobs not job * Add checkout * Fix parsing node version * Fix dashboard build on windows * Show codecov instead of coveralls
2019-10-28 21:57:33 +00:00
# -*- coding: utf-8 -*-
"""
proxy.py
~~~~~~~~
Fast, Lightweight, Programmable Proxy Server in a single Python file.
:copyright: (c) 2013-present by Abhinav Singh and contributors.
:license: BSD, see LICENSE for more details.
"""
import socket
import ssl
import logging
from abc import ABC, abstractmethod
from typing import NamedTuple, Optional, Union, Tuple
from ..common.constants import DEFAULT_BUFFER_SIZE
from ..common.utils import new_socket_connection
logger = logging.getLogger(__name__)
TcpConnectionTypes = NamedTuple('TcpConnectionTypes', [
('SERVER', int),
('CLIENT', int),
])
tcpConnectionTypes = TcpConnectionTypes(1, 2)
class TcpConnectionUninitializedException(Exception):
pass
class TcpConnection(ABC):
"""TCP server/client connection abstraction.
Main motivation of this class is to provide a buffer management
when reading and writing into the socket.
Implement the connection property abstract method to return
a socket connection object."""
def __init__(self, tag: int):
self.buffer: bytes = b''
self.closed: bool = False
self.tag: str = 'server' if tag == tcpConnectionTypes.SERVER else 'client'
@property
@abstractmethod
def connection(self) -> Union[ssl.SSLSocket, socket.socket]:
"""Must return the socket connection to use in this class."""
raise TcpConnectionUninitializedException() # pragma: no cover
def send(self, data: bytes) -> int:
"""Users must handle BrokenPipeError exceptions"""
return self.connection.send(data)
def recv(self, buffer_size: int = DEFAULT_BUFFER_SIZE) -> Optional[bytes]:
"""Users must handle socket.error exceptions"""
data: bytes = self.connection.recv(buffer_size)
if len(data) == 0:
return None
logger.debug(
'received %d bytes from %s' %
(len(data), self.tag))
# logger.info(data)
return data
def close(self) -> bool:
if not self.closed:
self.connection.close()
self.closed = True
return self.closed
def buffer_size(self) -> int:
return len(self.buffer)
def has_buffer(self) -> bool:
return self.buffer_size() > 0
def queue(self, data: bytes) -> int:
self.buffer += data
return len(data)
def flush(self) -> int:
"""Users must handle BrokenPipeError exceptions"""
if self.buffer_size() == 0:
return 0
sent: int = self.send(self.buffer)
# logger.info(self.buffer[:sent])
self.buffer = self.buffer[sent:]
logger.debug('flushed %d bytes to %s' % (sent, self.tag))
return sent
class TcpServerConnection(TcpConnection):
"""Establishes connection to upstream server."""
def __init__(self, host: str, port: int):
super().__init__(tcpConnectionTypes.SERVER)
self._conn: Optional[Union[ssl.SSLSocket, socket.socket]] = None
self.addr: Tuple[str, int] = (host, int(port))
@property
def connection(self) -> Union[ssl.SSLSocket, socket.socket]:
if self._conn is None:
raise TcpConnectionUninitializedException()
return self._conn
def connect(self) -> None:
if self._conn is not None:
return
self._conn = new_socket_connection(self.addr)
class TcpClientConnection(TcpConnection):
"""An accepted client connection request."""
def __init__(self,
conn: Union[ssl.SSLSocket, socket.socket],
addr: Tuple[str, int]):
super().__init__(tcpConnectionTypes.CLIENT)
self._conn: Optional[Union[ssl.SSLSocket, socket.socket]] = conn
self.addr: Tuple[str, int] = addr
@property
def connection(self) -> Union[ssl.SSLSocket, socket.socket]:
if self._conn is None:
raise TcpConnectionUninitializedException()
return self._conn