proxy.py/tutorial/requests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Http Requests\n",
    "\n",
    "## Usage\n",
    "\n",
    "To construct a HTTP request packet you have a variety of facilities available.\n",
    "\n",
    "Previously we saw how to parse HTTP responses using `HttpParser`.  We also saw how `HttpParser` class is capable of parsing various type of HTTP protocols.  Remember the _take away_ from that tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n'\n"
     ]
    }
   ],
   "source": [
    "from proxy.http.parser import HttpParser, httpParserTypes\n",
    "from proxy.http import httpMethods\n",
    "from proxy.common.utils import HTTP_1_1\n",
    "\n",
    "request = HttpParser(httpParserTypes.REQUEST_PARSER)\n",
    "request.path, request.method, request.version = b'/', httpMethods.GET, HTTP_1_1\n",
    "request.add_header(b'Host', b'jaxl.com')\n",
    "\n",
    "print(request.build())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But, this is a painful way to construct request packets.  Hence, other high level abstractions are available.\n",
    "\n",
    "Example, following one liner will give us the same request packet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\nUser-Agent: proxy.py v2.4.0rc10.dev13+g96428ae.d20220126\\r\\n\\r\\n'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from proxy.common.utils import build_http_request\n",
    "\n",
    "build_http_request(\n",
    "    method=httpMethods.GET,\n",
    "    url=b'/',\n",
    "    headers={b'Host': b'jaxl.com'},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`build_http_request` ensures a `User-Agent` header.  You can provide your own too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\nUser-Agent: my app v1\\r\\n\\r\\n'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "build_http_request(\n",
    "    method=httpMethods.GET,\n",
    "    url=b'/',\n",
    "    headers={\n",
    "        b'Host': b'jaxl.com',\n",
    "        b'User-Agent': b'my app v1'\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, if you don't want a `User-Agent` header at all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\n\\r\\n'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "build_http_request(\n",
    "    method=httpMethods.GET,\n",
    "    url=b'/',\n",
    "    headers={b'Host': b'jaxl.com'},\n",
    "    no_ua=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To add a connection close header, simply pass `conn_close=True`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'GET / HTTP/1.1\\r\\nHost: jaxl.com\\r\\nUser-Agent: proxy.py v2.4.0rc10.dev13+g96428ae.d20220126\\r\\nConnection: close\\r\\n\\r\\n'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "build_http_request(\n",
    "    method=httpMethods.GET,\n",
    "    url=b'/',\n",
    "    headers={b'Host': b'jaxl.com'},\n",
    "    conn_close=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For `POST` requests, provide the `body` attribute"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'POST / HTTP/1.1\\r\\nHost: jaxl.com\\r\\nContent-Type: application/x-www-form-urlencoded\\r\\nContent-Length: 21\\r\\nUser-Agent: proxy.py v2.4.0rc10.dev13+g96428ae.d20220126\\r\\nConnection: close\\r\\n\\r\\nkey=value&hello=world'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "build_http_request(\n",
    "    method=httpMethods.POST,\n",
    "    url=b'/',\n",
    "    headers={b'Host': b'jaxl.com'},\n",
    "    body=b'key=value&hello=world',\n",
    "    content_type=b'application/x-www-form-urlencoded',\n",
    "    conn_close=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For chunked data, simply include a `Transfer-Encoding` header.  This will omit the `Content-Length` header then:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'POST / HTTP/1.1\\r\\nHost: jaxl.com\\r\\nTransfer-Encoding: chunked\\r\\nContent-Type: application/x-www-form-urlencoded\\r\\nUser-Agent: proxy.py v2.4.0rc10.dev13+g96428ae.d20220126\\r\\nConnection: close\\r\\n\\r\\n5\\r\\nkey=v\\r\\n5\\r\\nalue&\\r\\n5\\r\\nhello\\r\\n5\\r\\n=worl\\r\\n1\\r\\nd\\r\\n0\\r\\n\\r\\n'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from proxy.http.parser import ChunkParser\n",
    "\n",
    "build_http_request(\n",
    "    method=httpMethods.POST,\n",
    "    url=b'/',\n",
    "    headers={\n",
    "        b'Host': b'jaxl.com',\n",
    "        b'Transfer-Encoding': b'chunked',\n",
    "    },\n",
    "    body=ChunkParser.to_chunks(b'key=value&hello=world', chunk_size=5),\n",
    "    content_type=b'application/x-www-form-urlencoded',\n",
    "    conn_close=True,\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "da9d6927d62b2b95bde149eedfbd5367cb7f465aad65a736f49c99ee3db39df7"
  },
  "kernelspec": {
   "display_name": "Python 3.10.0 64-bit ('venv310': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}