![](\"https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif\"){kind=logo}
\n",
"\n",
"[](https://pypi.org/project/tqdm)|[](https://tqdm.github.io/releases)|[](https://anaconda.org/conda-forge/tqdm)|[](https://hub.docker.com/r/tqdm/tqdm)|[](https://snapcraft.io/tqdm)\n",
"-|-|-|-|-\n",
"\n",
"[](https://github.com/tqdm/tqdm/actions?query=workflow%3ATest)|[](https://coveralls.io/github/tqdm/tqdm)|[](https://codecov.io/gh/tqdm/tqdm)|[](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[](https://libraries.io/pypi/tqdm)|[](https://pypi.org/project/tqdm)\n",
"-|-|-|-|-|-\n",
"\n",
"[](https://doi.org/10.5281/zenodo.595120)|[](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[](https://github.com/vinta/awesome-python)\n",
"-|-|-|-|-\n",
"\n",
"`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
"\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
"(*te quiero demasiado*).\n",
"\n",
"Instantly make your loops show a smart progress meter - just wrap any\n",
"iterable with `tqdm(iterable)`, and you're done!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 10000/10000 [00:00<00:00, 1541060.37it/s]\n"
]
}
],
"source": [
"from tqdm import tqdm\n",
"for i in tqdm(range(10000)):\n",
" pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`trange(N)` can be also used as a convenient shortcut for\n",
"`tqdm(xrange(N))`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"hello: 100%|██████████| 10.0k/10.0k [00:00<00:00, 1.56Mepoch/s]\n"
]
}
],
"source": [
"from tqdm import trange\n",
"for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
" pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"|[](https://tqdm.github.io/video) [](https://tqdm.github.io/PyData2019/slides.html)\n",
"-|-\n",
"\n",
"It can also be executed as a module with pipes:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"75.2MB [00:00, 217MB/s]\n",
"9999999\n"
]
}
],
"source": [
"! seq 9999999 | tqdm --bytes | wc -l"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```sh\n",
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
"is unit tested against performance regression. By comparison, the\n",
"well-established\n",
"[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
"an 800ns/iter overhead.\n",
"\n",
"In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
"the remaining time and to skip unnecessary iteration displays, which\n",
"allows for a negligible overhead in most cases.\n",
"\n",
"`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
"Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
"IPython/Jupyter notebooks.\n",
"\n",
"`tqdm` does not require any dependencies (not even `curses`!), just\n",
"Python and an environment supporting `carriage return \\r` and\n",
"`line feed \\n` control characters.\n",
"\n",
"---\n",
"\n",
"## Usage\n",
"\n",
"`tqdm` is very versatile and can be used in a number of ways.\n",
"The three main ones are given below.\n",
"\n",
"### Iterable-based\n",
"\n",
"Wrap `tqdm()` around any iterable:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tqdm import tqdm\n",
"from time import sleep"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 4/4 [00:01<00:00, 3.98it/s]\n"
]
}
],
"source": [
"text = \"\"\n",
"for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
" sleep(0.25)\n",
" text = text + char"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 96.76it/s]\n"
]
}
],
"source": [
"from tqdm import trange\n",
"\n",
"for i in trange(100):\n",
" sleep(0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Instantiation outside of the loop allows for manual control over `tqdm()`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Processing d: 100%|██████████| 4/4 [00:01<00:00, 3.90it/s]\n"
]
}
],
"source": [
"pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
"for char in pbar:\n",
" sleep(0.25)\n",
" pbar.set_description(\"Processing %s\" % char)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Manual\n",
"\n",
"Manual control of `tqdm()` updates using a `with` statement:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 97.81it/s]\n"
]
}
],
"source": [
"with tqdm(total=100) as pbar:\n",
" for i in range(10):\n",
" sleep(0.1)\n",
" pbar.update(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If the optional variable `total` (or an iterable with `len()`) is\n",
"provided, predictive stats are displayed.\n",
"\n",
"`with` is also optional (you can just assign `tqdm()` to a variable,\n",
"but in this case don't forget to `del` or `close()` at the end:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 97.85it/s]\n"
]
}
],
"source": [
"pbar = tqdm(total=100)\n",
"for i in range(10):\n",
" sleep(0.1)\n",
" pbar.update(10)\n",
"pbar.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Module"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
"command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
"pipes will pass through all `stdin` to `stdout` while printing progress\n",
"to `stderr`.\n",
"\n",
"The example below demonstrated counting the number of lines in all\n",
"Python files in the current directory, with timing information included."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5.80user 2.52system 0:08.86elapsed 93%CPU (0avgtext+0avgdata 3084maxresident)k\r\n",
"161232inputs+0outputs (0major+913287minor)pagefaults 0swaps\r\n",
"4104300\r\n"
]
}
],
"source": [
"! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"4043186it [00:05, 595767.35it/s]4.09user 1.23system 0:05.88elapsed 90%CPU (0avgtext+0avgdata 3092maxresident)k\n",
"20128inputs+0outputs (0major+904021minor)pagefaults 0swaps\n",
"4104300it [00:05, 698847.16it/s]\n",
"4104300\n"
]
}
],
"source": [
"! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the usual arguments for `tqdm` can also be specified."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████| 4.10M/4.10M [00:05<00:00, 724kloc/s]\n"
]
}
],
"source": [
"! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit_scale --total 4104300 >> /dev/null"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Backing up a large directory?\n",
"\n",
"```sh\n",
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
"```\n",
"\n",
"This can be beautified further:\n",
"\n",
"```sh\n",
"BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
"tar -cf - docs/ \\\n",
" | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
" | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
" > ~/backup.tgz\n",
"Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
"Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]\n",
"```\n",
"\n",
"Or done on a file level using 7-zip:\n",
"\n",
"```sh\n",
"7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
" | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
" | grep -v Compressing\n",
"100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Documentation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"tqdm?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Extra CLI Options\n",
" -----------------\n",
" delim : chr, optional\n",
" Delimiting character [default: '\\n']. Use '\\0' for null.\n",
" N.B.: on Windows systems, Python converts '\\n' to '\\r\\n'.\n",
" buf_size : int, optional\n",
" String buffer size in bytes [default: 256]\n",
" used when `delim` is specified.\n",
" bytes : bool, optional\n",
" If true, will count bytes, ignore `delim`, and default\n",
" `unit_scale` to True, `unit_divisor` to 1024, and `unit` to 'B'.\n",
" tee : bool, optional\n",
" If true, passes `stdin` to both `stderr` and `stdout`.\n",
" update : bool, optional\n",
" If true, will treat input as newly elapsed iterations,\n",
" i.e. numbers to pass to `update()`. Note that this is slow\n",
" (~2e5 it/s) since every input must be decoded as a number.\n",
" update_to : bool, optional\n",
" If true, will treat input as total elapsed iterations,\n",
" i.e. numbers to assign to `self.n`. Note that this is slow\n",
" (~2e5 it/s) since every input must be decoded as a number.\n",
" null : bool, optional\n",
" If true, will discard input (no stdout).\n",
" manpath : str, optional\n",
" Directory in which to install tqdm man pages.\n",
" comppath : str, optional\n",
" Directory in which to place tqdm completion.\n",
" log : str, optional\n",
" CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.\n"
]
}
],
"source": [
"! tqdm --help"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Examples and Advance Usage\n",
"\n",
"- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
" folder;\n",
"- import the module and run `help()`;\n",
"- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
" - this has an\n",
" [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
" on how to make a **great** progressbar;\n",
"- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
"- run this file!\n",
"\n",
"### Description and additional stats\n",
"\n",
"Custom information can be displayed and updated dynamically on `tqdm` bars\n",
"with the `desc` and `postfix` arguments:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GEN 9: 100%|██████████| 10/10 [00:01<00:00, 8.87it/s, gen=980, loss=0.762, lst=[1, 2], str=h]\n",
"Batch 4\n"
]
}
],
"source": [
"from tqdm import tqdm, trange\n",
"from random import random, randint\n",
"from time import sleep\n",
"\n",
"with trange(10) as t:\n",
" for i in t:\n",
" # Description will be displayed on the left\n",
" t.set_description('GEN %i' % i)\n",
" # Postfix will be displayed on the right,\n",
" # formatted automatically based on argument's datatype\n",
" t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
" lst=[1, 2])\n",
" sleep(0.1)\n",
"\n",
"with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
" postfix=[\"Batch\", dict(value=0)]) as t:\n",
" for i in range(10):\n",
" sleep(0.1)\n",
" t.postfix[1][\"value\"] = i / 2\n",
" t.update()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
"\n",
"- `postfix` also needs to be passed as an initial argument in a\n",
" compatible format, and\n",
"- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
" object. To prevent this behaviour, insert an extra item into the\n",
" dictionary where the key is not a string.\n",
"\n",
"Additional `bar_format` parameters may also be defined by overriding\n",
"`format_dict`, and the bar itself may be modified using `ascii`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"00:00 in total: 44%|0000. | 4/9 [00:00<00:00, 29799.67it/s]\n"
]
}
],
"source": [
"from tqdm import tqdm\n",
"from time import sleep\n",
"\n",
"class TqdmExtraFormat(tqdm):\n",
" \"\"\"Provides a `total_time` format parameter\"\"\"\n",
" @property\n",
" def format_dict(self):\n",
" d = super(TqdmExtraFormat, self).format_dict\n",
" total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
" d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
" return d\n",
"\n",
"for i in TqdmExtraFormat(\n",
" range(9), ascii=\" .oO0\",\n",
" bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
" if i == 4:\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that `{bar}` also supports a format specifier `[width][type]`.\n",
"\n",
"- `width`\n",
" + unspecified (default): automatic to fill `ncols`\n",
" + `int >= 0`: fixed width overriding `ncols` logic\n",
" + `int < 0`: subtract from the automatic default\n",
"- `type`\n",
" + `a`: ascii (`ascii=True` override)\n",
" + `u`: unicode (`ascii=False` override)\n",
" + `b`: blank (`ascii=\" \"` override)\n",
"\n",
"This means a fixed bar with right-justified text may be created by\n",
"using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Nested progress bars\n",
"\n",
"`tqdm` supports nested progress bars. Here's an example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tqdm.auto import trange\n",
"from time import sleep\n",
"\n",
"for i in trange(4, desc='1st loop'):\n",
" for j in trange(5, desc='2nd loop'):\n",
" for k in trange(50, desc='3rd loop', leave=False):\n",
" sleep(0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"On Windows [colorama](https://github.com/tartley/colorama) will be used\n",
"if available to keep nested bars on their respective lines.\n",
"\n",
"For manual control over positioning (e.g. for multi-processing use),\n",
"you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
"for the next, and so on. However, it's best to check if tqdm can work\n",
"without manual position first.\n",
"\n",
"```python\n",
"from time import sleep\n",
"from tqdm import trange, tqdm\n",
"from multiprocessing import Pool, RLock, freeze_support\n",
"\n",
"L = list(range(9))\n",
"\n",
"def progresser(n):\n",
" interval = 0.001 / (n + 2)\n",
" total = 5000\n",
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
" for _ in trange(total, desc=text, position=n):\n",
" sleep(interval)\n",
"\n",
"if __name__ == '__main__':\n",
" freeze_support() # for Windows support\n",
" tqdm.set_lock(RLock()) # for managing output contention\n",
" p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
" p.map(progresser, L)\n",
"```\n",
"\n",
"Note that in Python 3, `tqdm.write` is thread-safe:\n",
"\n",
"```python\n",
"from time import sleep\n",
"from tqdm import tqdm, trange\n",
"from concurrent.futures import ThreadPoolExecutor\n",
"\n",
"L = list(range(9))\n",
"\n",
"def progresser(n):\n",
" interval = 0.001 / (n + 2)\n",
" total = 5000\n",
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
" for _ in trange(total, desc=text):\n",
" sleep(interval).auto\n",
" if n == 6:\n",
" tqdm.write(\"n == 6 completed.\")\n",
" tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
"\n",
"if __name__ == '__main__':\n",
" with ThreadPoolExecutor() as p:\n",
" p.map(progresser, L)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Hooks and callbacks\n",
"\n",
"`tqdm` can easily support callbacks/hooks and manual updates.\n",
"Here's an example with `urllib`:\n",
"\n",
"**`urllib.urlretrieve` documentation**\n",
"\n",
"> [...]\n",
"> If present, the hook function will be called once\n",
"> on establishment of the network connection and once after each block read\n",
"> thereafter. The hook will be passed three arguments; a count of blocks\n",
"> transferred so far, a block size in bytes, and the total size of the file.\n",
"> [...]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"matryoshka.zip: 100%|██████████| 262k/262k [00:02<00:00, 87.9kB/s] \n"
]
}
],
"source": [
"import urllib, os\n",
"from tqdm import tqdm\n",
"urllib = getattr(urllib, 'request', urllib)\n",
"\n",
"class TqdmUpTo(tqdm):\n",
" \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
" def update_to(self, b=1, bsize=1, tsize=None):\n",
" \"\"\"\n",
" b : int, optional\n",
" Number of blocks transferred so far [default: 1].\n",
" bsize : int, optional\n",
" Size of each block (in tqdm units) [default: 1].\n",
" tsize : int, optional\n",
" Total size (in tqdm units). If [default: None] remains unchanged.\n",
" \"\"\"\n",
" if tsize is not None:\n",
" self.total = tsize\n",
" return self.update(b * bsize - self.n) # also sets self.n = b * bsize\n",
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
"with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n",
" desc=eg_link.split('/')[-1]) as t: # all optional kwargs\n",
" urllib.urlretrieve(eg_link, filename=os.devnull,\n",
" reporthook=t.update_to, data=None)\n",
" t.total = t.n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
"Functional alternative in\n",
"[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
"\n",
"It is recommend to use `miniters=1` whenever there is potentially large\n",
"differences in iteration speed (e.g. downloading a file over a patchy\n",
"connection).\n",
"\n",
"**Wrapping read/write methods**\n",
"\n",
"To measure throughput through a file-like object's `read` or `write`\n",
"methods, use `CallbackIOWrapper`:\n",
"\n",
"```python\n",
"from tqdm import tqdm\n",
"from tqdm.utils import CallbackIOWrapper\n",
"\n",
"with tqdm(total=file_obj.size,\n",
" unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
" fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
" while True:\n",
" chunk = fobj.read(chunk_size)\n",
" if not chunk:\n",
" break\n",
" t.reset()\n",
" # ... continue to use `t` for something else\n",
"```\n",
"\n",
"Alternatively, use the even simpler `wrapattr` convenience function,\n",
"which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
"down to:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"matryoshka.zip: 100%|██████████| 254k/254k [00:00<00:00, 602kB/s] \n"
]
}
],
"source": [
"import urllib, os\n",
"from tqdm import tqdm\n",
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
"response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n",
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
" miniters=1, desc=eg_link.split('/')[-1],\n",
" total=getattr(response, 'length', None)) as fout:\n",
" for chunk in response:\n",
" fout.write(chunk)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `requests` equivalent is nearly identical:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"matryoshka.zip: 254kB [00:00, 460kB/s] \n"
]
}
],
"source": [
"import requests, os\n",
"from tqdm import tqdm\n",
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
"response = requests.get(eg_link, stream=True)\n",
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
" miniters=1, desc=eg_link.split('/')[-1],\n",
" total=int(response.headers.get('content-length', 0))) as fout:\n",
" for chunk in response.iter_content(chunk_size=4096):\n",
" fout.write(chunk)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Pandas Integration\n",
"\n",
"Due to popular demand we've added support for `pandas` -- here's an example\n",
"for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"my bar!: 100%|██████████| 6/6 [00:00<00:00, 193.96it/s]\n"
]
},
{
"data": {
"text/html": [
"| \n", " | 0 | \n", "1 | \n", "2 | \n", "3 | \n", "4 | \n", "5 | \n", "
|---|---|---|---|---|---|---|
| 0 | \n", "1296 | \n", "1156 | \n", "961 | \n", "576 | \n", "6889 | \n", "7921 | \n", "
| 1 | \n", "225 | \n", "3844 | \n", "8649 | \n", "121 | \n", "8281 | \n", "4624 | \n", "
| 2 | \n", "3025 | \n", "729 | \n", "9025 | \n", "4225 | \n", "1024 | \n", "8836 | \n", "
| 3 | \n", "25 | \n", "729 | \n", "1936 | \n", "6724 | \n", "4900 | \n", "3721 | \n", "
| 4 | \n", "7225 | \n", "1156 | \n", "64 | \n", "9604 | \n", "441 | \n", "961 | \n", "
| 5 | \n", "5184 | \n", "441 | \n", "2500 | \n", "5041 | \n", "1936 | \n", "5041 | \n", "
| 6 | \n", "289 | \n", "8281 | \n", "8464 | \n", "1089 | \n", "7569 | \n", "25 | \n", "
| 7 | \n", "7225 | \n", "1600 | \n", "5625 | \n", "5625 | \n", "3364 | \n", "7396 | \n", "
| 8 | \n", "3025 | \n", "4489 | \n", "2704 | \n", "400 | \n", "6724 | \n", "3364 | \n", "
| 9 | \n", "6400 | \n", "7396 | \n", "4225 | \n", "49 | \n", "7921 | \n", "4761 | \n", "
| 10 | \n", "64 | \n", "256 | \n", "4356 | \n", "8100 | \n", "4761 | \n", "4761 | \n", "
| 11 | \n", "3600 | \n", "729 | \n", "4624 | \n", "2500 | \n", "25 | \n", "441 | \n", "
| 12 | \n", "5476 | \n", "5329 | \n", "6084 | \n", "3025 | \n", "3844 | \n", "7396 | \n", "
| 13 | \n", "16 | \n", "6889 | \n", "400 | \n", "484 | \n", "1225 | \n", "169 | \n", "
| 14 | \n", "3025 | \n", "2209 | \n", "9801 | \n", "324 | \n", "4900 | \n", "4096 | \n", "
| 15 | \n", "8649 | \n", "6400 | \n", "1936 | \n", "3600 | \n", "4900 | \n", "2116 | \n", "
| 16 | \n", "4356 | \n", "3481 | \n", "5476 | \n", "7569 | \n", "9216 | \n", "16 | \n", "
| 17 | \n", "961 | \n", "196 | \n", "4096 | \n", "5476 | \n", "8649 | \n", "6724 | \n", "
| 18 | \n", "1 | \n", "4900 | \n", "1156 | \n", "2025 | \n", "2916 | \n", "1521 | \n", "
| 19 | \n", "6889 | \n", "2809 | \n", "9409 | \n", "3249 | \n", "7569 | \n", "784 | \n", "
| 20 | \n", "2116 | \n", "49 | \n", "225 | \n", "81 | \n", "8649 | \n", "289 | \n", "
| 21 | \n", "7569 | \n", "2500 | \n", "25 | \n", "441 | \n", "9025 | \n", "6561 | \n", "
| 22 | \n", "1 | \n", "4489 | \n", "3136 | \n", "3025 | \n", "3249 | \n", "2116 | \n", "
| 23 | \n", "5476 | \n", "2116 | \n", "9025 | \n", "4096 | \n", "1764 | \n", "16 | \n", "
| 24 | \n", "3025 | \n", "961 | \n", "3249 | \n", "7056 | \n", "3136 | \n", "676 | \n", "
| 25 | \n", "2809 | \n", "7744 | \n", "1089 | \n", "6724 | \n", "0 | \n", "961 | \n", "
| 26 | \n", "225 | \n", "400 | \n", "7396 | \n", "3721 | \n", "784 | \n", "7744 | \n", "
| 27 | \n", "81 | \n", "100 | \n", "400 | \n", "8836 | \n", "2601 | \n", "1600 | \n", "
| 28 | \n", "441 | \n", "144 | \n", "2704 | \n", "1369 | \n", "289 | \n", "8281 | \n", "
| 29 | \n", "625 | \n", "3025 | \n", "8836 | \n", "64 | \n", "3025 | \n", "9409 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 99970 | \n", "9216 | \n", "8100 | \n", "1369 | \n", "529 | \n", "625 | \n", "7921 | \n", "
| 99971 | \n", "4 | \n", "5184 | \n", "3364 | \n", "4356 | \n", "5625 | \n", "9604 | \n", "
| 99972 | \n", "484 | \n", "6889 | \n", "900 | \n", "36 | \n", "1444 | \n", "3364 | \n", "
| 99973 | \n", "7744 | \n", "3844 | \n", "4489 | \n", "5776 | \n", "289 | \n", "1225 | \n", "
| 99974 | \n", "1225 | \n", "4489 | \n", "9 | \n", "2916 | \n", "1369 | \n", "100 | \n", "
| 99975 | \n", "1849 | \n", "4356 | \n", "169 | \n", "2025 | \n", "2704 | \n", "7744 | \n", "
| 99976 | \n", "5929 | \n", "7396 | \n", "841 | \n", "441 | \n", "225 | \n", "7056 | \n", "
| 99977 | \n", "676 | \n", "7056 | \n", "529 | \n", "100 | \n", "16 | \n", "484 | \n", "
| 99978 | \n", "841 | \n", "5476 | \n", "8100 | \n", "7569 | \n", "6400 | \n", "1156 | \n", "
| 99979 | \n", "4 | \n", "7921 | \n", "1225 | \n", "5476 | \n", "3844 | \n", "7396 | \n", "
| 99980 | \n", "6241 | \n", "1444 | \n", "8281 | \n", "1849 | \n", "5625 | \n", "3721 | \n", "
| 99981 | \n", "2401 | \n", "121 | \n", "169 | \n", "49 | \n", "2916 | \n", "1296 | \n", "
| 99982 | \n", "1 | \n", "1764 | \n", "441 | \n", "8836 | \n", "6561 | \n", "169 | \n", "
| 99983 | \n", "1444 | \n", "3600 | \n", "9025 | \n", "8281 | \n", "7569 | \n", "1444 | \n", "
| 99984 | \n", "5776 | \n", "1600 | \n", "9 | \n", "9409 | \n", "961 | \n", "3249 | \n", "
| 99985 | \n", "2916 | \n", "4356 | \n", "64 | \n", "1681 | \n", "49 | \n", "1936 | \n", "
| 99986 | \n", "5929 | \n", "1089 | \n", "169 | \n", "4225 | \n", "4225 | \n", "1296 | \n", "
| 99987 | \n", "3481 | \n", "2116 | \n", "5041 | \n", "7569 | \n", "6724 | \n", "25 | \n", "
| 99988 | \n", "3136 | \n", "4225 | \n", "676 | \n", "484 | \n", "961 | \n", "4624 | \n", "
| 99989 | \n", "2500 | \n", "7056 | \n", "900 | \n", "4096 | \n", "3844 | \n", "8100 | \n", "
| 99990 | \n", "784 | \n", "900 | \n", "2916 | \n", "4356 | \n", "7396 | \n", "2916 | \n", "
| 99991 | \n", "3721 | \n", "1764 | \n", "16 | \n", "9 | \n", "5929 | \n", "4900 | \n", "
| 99992 | \n", "64 | \n", "2809 | \n", "3025 | \n", "9 | \n", "5625 | \n", "484 | \n", "
| 99993 | \n", "2116 | \n", "1600 | \n", "5929 | \n", "81 | \n", "625 | \n", "441 | \n", "
| 99994 | \n", "1521 | \n", "529 | \n", "2025 | \n", "2809 | \n", "3364 | \n", "5929 | \n", "
| 99995 | \n", "576 | \n", "1225 | \n", "1764 | \n", "3721 | \n", "0 | \n", "144 | \n", "
| 99996 | \n", "625 | \n", "5476 | \n", "3364 | \n", "9409 | \n", "100 | \n", "4900 | \n", "
| 99997 | \n", "900 | \n", "1444 | \n", "1521 | \n", "676 | \n", "49 | \n", "7056 | \n", "
| 99998 | \n", "6084 | \n", "36 | \n", "289 | \n", "6561 | \n", "7921 | \n", "256 | \n", "
| 99999 | \n", "1089 | \n", "784 | \n", "3969 | \n", "2500 | \n", "2401 | \n", "2500 | \n", "
100000 rows × 6 columns
\n", "