proxy.py/examples
Abhinav Singh 2fa320d03f
Python 3.11 support (#1384)
* Changes for Python 3.11 support

* Updated README.md for versioning info

* Update `httpx==0.27.0` to avoid `cgi` deprecation warning from pytest on Python 3.11

* Make tests work for 3.11

* Declare support for 3.11

* Use 3.11-alpine for Docker images

* Preserve pylint version for `python_version <= 3.10`

* Preserve httpx version for <= 3.10

* `httpx` usage fix in tests for <=3.10

* Adjust pylint and pytest for >= 3.11

* Use 3.11.8, bad-option-value and httpx proxies fix

* tox for 3.11

* Fix for `TOXENV: py`

* -vv for pytest

* Downgrade to `pytest-asyncio==0.21.1`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove asyncio_mode=strict

* try with `pytest-cov==4.1.0` for 3.11

* bump coverage for 3.11

* Try `3.11` in GitHub workflow which installs >3.11.8 unavailable via pyenv yet

* Revert back to `-v`

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-04-13 13:22:25 +05:30
..
README.md Add `benchmark` results to `README.md` (#828) 2021-11-30 00:10:04 +05:30
https_connect_tunnel.py `isort` everything except lib (for now) (#952) 2022-01-10 13:28:22 +05:30
pubsub_eventing.py [Examples] Fix broken examples (#1077) 2022-01-30 18:30:54 +05:30
ssl_echo_client.py [Examples] Fix broken examples (#1077) 2022-01-30 18:30:54 +05:30
ssl_echo_server.py [Examples] Fix broken examples (#1077) 2022-01-30 18:30:54 +05:30
task.py [Task] A generic payload based work abstraction (#1057) 2022-01-25 01:51:41 +05:30
tcp_echo_client.py [Examples] Fix broken examples (#1077) 2022-01-30 18:30:54 +05:30
tcp_echo_server.py [Work] `kwargs` independent `work_klass` creation and work core (#1051) 2022-01-24 03:16:08 +05:30
web_scraper.py Python 3.11 support (#1384) 2024-04-13 13:22:25 +05:30
websocket_client.py [Examples] Fix broken examples (#1077) 2022-01-30 18:30:54 +05:30

README.md

Proxy Library Examples

This directory contains examples that demonstrate proxy.py core library capabilities.

Looking for proxy.py plugin examples? Check proxy/plugin directory.

Table of Contents

WebSocket Client

  1. Makes use of proxy.http.websocket.WebsocketClient which is built on-top of asyncio
  2. websocket_client.py by default opens a WebSocket connection to ws://echo.websocket.org.
  3. Client will exchange num_echos = 10 packets with the server and then shutdown.

Start websocket_client.py as:

 PYTHONPATH=. python examples/websocket_client.py
Received b'hello' after 306 millisec
Received b'hello' after 308 millisec
Received b'hello' after 277 millisec
Received b'hello' after 334 millisec
Received b'hello' after 296 millisec
Received b'hello' after 317 millisec
Received b'hello' after 307 millisec
Received b'hello' after 307 millisec
Received b'hello' after 306 millisec
Received b'hello' after 307 millisec
Received b'hello' after 309 millisec

TCP Echo Server

  1. Makes use of proxy.core.acceptor.AcceptorPool, same multicore acceptor used internally by proxy.py server.
  2. Implements proxy.core.acceptor.Work interface to handle incoming client connections.

Start tcp_echo_server.py as:

 PYTHONPATH=. python examples/tcp_echo_server.py
Connection accepted from ('::1', 53285, 0, 0)
Connection closed by client ('::1', 53285, 0, 0)

TCP Echo Client

  1. Makes use of proxy.common.utils.socket_connection to establish a TCP socket connection with our TCP echo server.
  2. Exchanges packet with server in an infinite loop. Press CTRL+C to stop.

Start tcp_echo_client.py as:

 PYTHONPATH=. python examples/tcp_echo_client.py
b'hello'
b'hello'
b'hello'
b'hello'
b'hello'
...
...
...
^CTraceback (most recent call last):
  File "examples/tcp_echo_client.py", line 18, in <module>
    data = client.recv(DEFAULT_BUFFER_SIZE)
KeyboardInterrupt

SSL Echo Server

  1. Same as tcp_echo_server.py.
  2. Internally uses proxy.common.utils.wrap_socket to enable SSL encryption.
  3. Uses https-key.pem and https-signed-cert.pem for SSL encryption. See End-to-End Encryption for instructions on how to generate SSL certificates.

Start ssl_echo_server.py as:

 PYTHONPATH=. python examples/ssl_echo_server.py

SSL Echo Client

  1. Makes use of proxy.core.connection.TcpServerConnection to establish a SSL connection with our ssl_echo_server.py.
  2. Uses generated ca-cert.pem for SSL certificate verification.

Start ssl_echo_client.py as:

 PYTHONPATH=. python examples/ssl_echo_client.py

PubSub Eventing

  1. Makes use of proxy.py core eventing module.
  2. A proxy.core.event.EventDispatcher thread is started.
  3. A proxy.core.event.EventSubscriber thread is started.
  4. A multiprocessing.Process publisher is started.
  5. Main thread also publishes into EventDispatcher queue.
  6. Events from both the main thread and another process are received by the subscriber.

Start pubsub_eventing.py as:

 PYTHONPATH=. python examples/pubsub_eventing.py
DEBUG:proxy.core.event.subscriber:Subscribed relay sub id 5eb22010764f4d44900f41e2fb408ca6 from core events
publisher starting
^Cpublisher shutdown
bye!!!
DEBUG:proxy.core.event.subscriber:Un-subscribed relay sub id 5eb22010764f4d44900f41e2fb408ca6 from core events
Received 52724 events from main thread, 60172 events from another process, in 21.50117802619934 seconds

HTTPS Connect Tunnel

A simple HTTP proxy server supporting only CONNECT (https) requests.

  1. Uses HttpParser for request parsing.
  2. Uses TcpServerConnection to establish upstream connection.
  3. Overrides BaseServer methods to also register read/write events for upstream connection.

Start https_connect_tunnel.py as:

 PYTHONPATH=. python examples/https_connect_tunnel.py

Send https requests via tunnel as:

 curl -x localhost:12345 https://httpbin.org/get

Generic Work Acceptor and Executor

  1. Makes use of proxy.core.AcceptorPool and proxy.core.Work
  2. Demonstrates how to perform generic work using proxy.py core.

Start web_scraper.py as:

 PYTHONPATH=. python examples/web_scraper.py