2018-12-18 20:24:33 +00:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"<h1 align=\"center\">tqdm</h1>\n",
2019-01-19 00:12:19 +00:00
"<img src=\"https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif\" align=\"left\" />\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"[](https://pypi.org/project/tqdm)|[](https://tqdm.github.io/releases)|[](https://anaconda.org/conda-forge/tqdm)|[](https://hub.docker.com/r/tqdm/tqdm)|[](https://snapcraft.io/tqdm)\n",
2019-05-19 20:01:23 +00:00
"-|-|-|-|-\n",
2019-01-18 23:28:10 +00:00
"\n",
2021-02-10 20:24:48 +00:00
"[](https://github.com/tqdm/tqdm/actions?query=workflow%3ATest)|[](https://coveralls.io/github/tqdm/tqdm)|[](https://codecov.io/gh/tqdm/tqdm)|[](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[](https://libraries.io/pypi/tqdm)|[](https://pypi.org/project/tqdm)\n",
2019-01-18 23:28:10 +00:00
"-|-|-|-|-|-\n",
"\n",
2021-02-10 20:24:48 +00:00
"[](https://doi.org/10.5281/zenodo.595120)|[](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[](https://github.com/vinta/awesome-python)\n",
2020-12-24 23:40:58 +00:00
"-|-|-|-|-\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
"\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
"(*te quiero demasiado*).\n",
2019-01-18 23:28:10 +00:00
"\n",
"Instantly make your loops show a smart progress meter - just wrap any\n",
"iterable with `tqdm(iterable)`, and you're done!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 10000/10000 [00:00<00:00, 1541060.37it/s]\n"
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"from tqdm import tqdm\n",
"for i in tqdm(range(10000)):\n",
" pass"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"`trange(N)` can be also used as a convenient shortcut for\n",
"`tqdm(xrange(N))`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"hello: 100%|██████████| 10.0k/10.0k [00:00<00:00, 1.56Mepoch/s]\n"
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"from tqdm import trange\n",
"for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
" pass"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2021-01-12 13:58:24 +00:00
"|[](https://tqdm.github.io/video) [](https://tqdm.github.io/PyData2019/slides.html)\n",
2020-03-29 23:13:41 +00:00
"-|-\n",
2019-01-18 23:28:10 +00:00
"\n",
"It can also be executed as a module with pipes:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"75.2MB [00:00, 217MB/s]\n",
"9999999\n"
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"! seq 9999999 | tqdm --bytes | wc -l"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"```sh\n",
2020-03-29 23:13:41 +00:00
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
2019-01-18 23:28:10 +00:00
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
2019-01-18 23:28:10 +00:00
"is unit tested against performance regression. By comparison, the\n",
"well-established\n",
"[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
"an 800ns/iter overhead.\n",
"\n",
"In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
"the remaining time and to skip unnecessary iteration displays, which\n",
"allows for a negligible overhead in most cases.\n",
"\n",
"`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
"Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
"IPython/Jupyter notebooks.\n",
"\n",
"`tqdm` does not require any dependencies (not even `curses`!), just\n",
"Python and an environment supporting `carriage return \\r` and\n",
2019-01-18 23:55:05 +00:00
"`line feed \\n` control characters.\n",
"\n",
"---\n",
"\n",
2018-12-18 20:24:33 +00:00
"## Usage\n",
"\n",
2019-01-18 23:28:10 +00:00
"`tqdm` is very versatile and can be used in a number of ways.\n",
2019-01-18 23:55:05 +00:00
"The three main ones are given below.\n",
"\n",
2018-12-18 20:24:33 +00:00
"### Iterable-based\n",
"\n",
"Wrap `tqdm()` around any iterable:"
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
"outputs": [],
"source": [
2019-01-18 23:28:10 +00:00
"from tqdm import tqdm\n",
2020-03-29 23:13:41 +00:00
"from time import sleep"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 4/4 [00:01<00:00, 3.98it/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
"text = \"\"\n",
"for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
2020-03-29 23:13:41 +00:00
" sleep(0.25)\n",
2018-12-18 20:24:33 +00:00
" text = text + char"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 96.76it/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
2018-12-18 20:36:39 +00:00
"from tqdm import trange\n",
"\n",
2019-01-18 23:28:10 +00:00
"for i in trange(100):\n",
2020-03-29 23:13:41 +00:00
" sleep(0.01)"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Instantiation outside of the loop allows for manual control over `tqdm()`:"
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Processing d: 100%|██████████| 4/4 [00:01<00:00, 3.90it/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
"pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
"for char in pbar:\n",
2020-03-29 23:13:41 +00:00
" sleep(0.25)\n",
2018-12-18 20:24:33 +00:00
" pbar.set_description(\"Processing %s\" % char)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Manual\n",
"\n",
2020-03-29 23:13:41 +00:00
"Manual control of `tqdm()` updates using a `with` statement:"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 97.81it/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
"with tqdm(total=100) as pbar:\n",
" for i in range(10):\n",
2020-03-29 23:13:41 +00:00
" sleep(0.1)\n",
2018-12-18 20:24:33 +00:00
" pbar.update(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"If the optional variable `total` (or an iterable with `len()`) is\n",
"provided, predictive stats are displayed.\n",
2018-12-18 20:24:33 +00:00
"\n",
2019-01-18 23:28:10 +00:00
"`with` is also optional (you can just assign `tqdm()` to a variable,\n",
"but in this case don't forget to `del` or `close()` at the end:"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 100/100 [00:01<00:00, 97.85it/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
"pbar = tqdm(total=100)\n",
"for i in range(10):\n",
2020-03-29 23:13:41 +00:00
" sleep(0.1)\n",
2018-12-18 20:24:33 +00:00
" pbar.update(10)\n",
"pbar.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"### Module"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
"command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
"pipes will pass through all `stdin` to `stdout` while printing progress\n",
"to `stderr`.\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"The example below demonstrated counting the number of lines in all\n",
"Python files in the current directory, with timing information included."
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"5.80user 2.52system 0:08.86elapsed 93%CPU (0avgtext+0avgdata 3084maxresident)k\r\n",
"161232inputs+0outputs (0major+913287minor)pagefaults 0swaps\r\n",
"4104300\r\n"
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"4043186it [00:05, 595767.35it/s]4.09user 1.23system 0:05.88elapsed 90%CPU (0avgtext+0avgdata 3092maxresident)k\n",
"20128inputs+0outputs (0major+904021minor)pagefaults 0swaps\n",
"4104300it [00:05, 698847.16it/s]\n",
"4104300\n"
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the usual arguments for `tqdm` can also be specified."
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████| 4.10M/4.10M [00:05<00:00, 724kloc/s]\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
2019-01-18 23:28:10 +00:00
"! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit_scale --total 4104300 >> /dev/null"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"Backing up a large directory?\n",
"\n",
2019-01-18 23:28:10 +00:00
"```sh\n",
2020-03-29 23:13:41 +00:00
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
"```\n",
"\n",
"This can be beautified further:\n",
"\n",
"```sh\n",
"BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
"tar -cf - docs/ \\\n",
" | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
" | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
" > ~/backup.tgz\n",
"Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
"Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]\n",
"```\n",
"\n",
"Or done on a file level using 7-zip:\n",
"\n",
"```sh\n",
"7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
" | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
" | grep -v Compressing\n",
"100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
2019-01-18 23:28:10 +00:00
"```"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"## Documentation"
]
},
{
"cell_type": "code",
"execution_count": null,
2019-01-18 23:55:05 +00:00
"metadata": {},
2019-01-18 23:28:10 +00:00
"outputs": [],
"source": [
2019-01-18 23:55:05 +00:00
"tqdm?"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
2019-01-18 23:55:05 +00:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Extra CLI Options\n",
" -----------------\n",
" delim : chr, optional\n",
" Delimiting character [default: '\\n']. Use '\\0' for null.\n",
" N.B.: on Windows systems, Python converts '\\n' to '\\r\\n'.\n",
" buf_size : int, optional\n",
" String buffer size in bytes [default: 256]\n",
" used when `delim` is specified.\n",
" bytes : bool, optional\n",
" If true, will count bytes, ignore `delim`, and default\n",
" `unit_scale` to True, `unit_divisor` to 1024, and `unit` to 'B'.\n",
2021-02-10 20:24:48 +00:00
" tee : bool, optional\n",
" If true, passes `stdin` to both `stderr` and `stdout`.\n",
" update : bool, optional\n",
" If true, will treat input as newly elapsed iterations,\n",
" i.e. numbers to pass to `update()`. Note that this is slow\n",
" (~2e5 it/s) since every input must be decoded as a number.\n",
" update_to : bool, optional\n",
" If true, will treat input as total elapsed iterations,\n",
" i.e. numbers to assign to `self.n`. Note that this is slow\n",
" (~2e5 it/s) since every input must be decoded as a number.\n",
" null : bool, optional\n",
" If true, will discard input (no stdout).\n",
2019-01-18 23:55:05 +00:00
" manpath : str, optional\n",
" Directory in which to install tqdm man pages.\n",
2020-05-02 23:04:23 +00:00
" comppath : str, optional\n",
" Directory in which to place tqdm completion.\n",
2019-01-18 23:55:05 +00:00
" log : str, optional\n",
2021-02-10 20:24:48 +00:00
" CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.\n"
2019-01-18 23:55:05 +00:00
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
2019-01-18 23:55:05 +00:00
"! tqdm --help"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Examples and Advance Usage\n",
"\n",
2019-01-19 00:12:19 +00:00
"- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
2019-01-18 23:28:10 +00:00
" folder;\n",
"- import the module and run `help()`;\n",
"- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
" - this has an\n",
" [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
2020-03-29 23:13:41 +00:00
" on how to make a **great** progressbar;\n",
"- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
2019-01-18 23:28:10 +00:00
"- run this file!\n",
"\n",
2018-12-18 20:24:33 +00:00
"### Description and additional stats\n",
"\n",
2019-01-18 23:28:10 +00:00
"Custom information can be displayed and updated dynamically on `tqdm` bars\n",
"with the `desc` and `postfix` arguments:"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GEN 9: 100%|██████████| 10/10 [00:01<00:00, 8.87it/s, gen=980, loss=0.762, lst=[1, 2], str=h]\n",
"Batch 4\n"
]
}
],
2018-12-18 20:24:33 +00:00
"source": [
2020-03-29 23:13:41 +00:00
"from tqdm import tqdm, trange\n",
2018-12-18 20:24:33 +00:00
"from random import random, randint\n",
"from time import sleep\n",
"\n",
2019-01-18 23:28:10 +00:00
"with trange(10) as t:\n",
2018-12-18 20:24:33 +00:00
" for i in t:\n",
" # Description will be displayed on the left\n",
" t.set_description('GEN %i' % i)\n",
" # Postfix will be displayed on the right,\n",
" # formatted automatically based on argument's datatype\n",
" t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
" lst=[1, 2])\n",
" sleep(0.1)\n",
"\n",
"with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
" postfix=[\"Batch\", dict(value=0)]) as t:\n",
" for i in range(10):\n",
" sleep(0.1)\n",
" t.postfix[1][\"value\"] = i / 2\n",
" t.update()"
]
},
2019-01-18 23:28:10 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
"\n",
"- `postfix` also needs to be passed as an initial argument in a\n",
" compatible format, and\n",
"- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
" object. To prevent this behaviour, insert an extra item into the\n",
" dictionary where the key is not a string.\n",
"\n",
"Additional `bar_format` parameters may also be defined by overriding\n",
"`format_dict`, and the bar itself may be modified using `ascii`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2021-02-10 20:24:48 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"00:00 in total: 44%|0000. | 4/9 [00:00<00:00, 29799.67it/s]\n"
]
}
],
2020-03-29 23:13:41 +00:00
"source": [
"from tqdm import tqdm\n",
"from time import sleep\n",
"\n",
"class TqdmExtraFormat(tqdm):\n",
" \"\"\"Provides a `total_time` format parameter\"\"\"\n",
" @property\n",
" def format_dict(self):\n",
" d = super(TqdmExtraFormat, self).format_dict\n",
" total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
" d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
" return d\n",
"\n",
"for i in TqdmExtraFormat(\n",
" range(9), ascii=\" .oO0\",\n",
" bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
" if i == 4:\n",
" break"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that `{bar}` also supports a format specifier `[width][type]`.\n",
"\n",
"- `width`\n",
" + unspecified (default): automatic to fill `ncols`\n",
" + `int >= 0`: fixed width overriding `ncols` logic\n",
" + `int < 0`: subtract from the automatic default\n",
"- `type`\n",
" + `a`: ascii (`ascii=True` override)\n",
" + `u`: unicode (`ascii=False` override)\n",
" + `b`: blank (`ascii=\" \"` override)\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"This means a fixed bar with right-justified text may be created by\n",
"using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
2019-01-18 23:28:10 +00:00
]
},
2018-12-18 20:24:33 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Nested progress bars\n",
"\n",
"`tqdm` supports nested progress bars. Here's an example:"
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:28:10 +00:00
"outputs": [],
2018-12-18 20:24:33 +00:00
"source": [
2019-01-18 23:28:10 +00:00
"from tqdm.auto import trange\n",
2018-12-18 20:24:33 +00:00
"from time import sleep\n",
"\n",
2019-01-18 23:28:10 +00:00
"for i in trange(4, desc='1st loop'):\n",
" for j in trange(5, desc='2nd loop'):\n",
2020-03-29 23:13:41 +00:00
" for k in trange(50, desc='3rd loop', leave=False):\n",
2019-01-18 23:28:10 +00:00
" sleep(0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"On Windows [colorama](https://github.com/tartley/colorama) will be used\n",
"if available to keep nested bars on their respective lines.\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"For manual control over positioning (e.g. for multi-processing use),\n",
"you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
"for the next, and so on. However, it's best to check if tqdm can work\n",
"without manual position first.\n",
"\n",
"```python\n",
2019-01-18 23:28:10 +00:00
"from time import sleep\n",
"from tqdm import trange, tqdm\n",
2020-10-07 22:46:13 +00:00
"from multiprocessing import Pool, RLock, freeze_support\n",
2019-01-18 23:28:10 +00:00
"\n",
"L = list(range(9))\n",
"\n",
"def progresser(n):\n",
" interval = 0.001 / (n + 2)\n",
" total = 5000\n",
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
2020-03-29 23:13:41 +00:00
" for _ in trange(total, desc=text, position=n):\n",
2019-01-18 23:28:10 +00:00
" sleep(interval)\n",
"\n",
"if __name__ == '__main__':\n",
" freeze_support() # for Windows support\n",
2020-10-07 22:46:13 +00:00
" tqdm.set_lock(RLock()) # for managing output contention\n",
2020-03-29 23:13:41 +00:00
" p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
2019-01-18 23:28:10 +00:00
" p.map(progresser, L)\n",
2020-03-29 23:13:41 +00:00
"```\n",
"\n",
"Note that in Python 3, `tqdm.write` is thread-safe:\n",
"\n",
"```python\n",
"from time import sleep\n",
"from tqdm import tqdm, trange\n",
"from concurrent.futures import ThreadPoolExecutor\n",
"\n",
"L = list(range(9))\n",
"\n",
"def progresser(n):\n",
" interval = 0.001 / (n + 2)\n",
" total = 5000\n",
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
" for _ in trange(total, desc=text):\n",
" sleep(interval).auto\n",
" if n == 6:\n",
" tqdm.write(\"n == 6 completed.\")\n",
" tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
"\n",
"if __name__ == '__main__':\n",
" with ThreadPoolExecutor() as p:\n",
" p.map(progresser, L)\n",
"```"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Hooks and callbacks\n",
"\n",
"`tqdm` can easily support callbacks/hooks and manual updates.\n",
"Here's an example with `urllib`:\n",
"\n",
2020-03-29 23:13:41 +00:00
"**`urllib.urlretrieve` documentation**\n",
2019-01-18 23:28:10 +00:00
"\n",
"> [...]\n",
"> If present, the hook function will be called once\n",
"> on establishment of the network connection and once after each block read\n",
"> thereafter. The hook will be passed three arguments; a count of blocks\n",
"> transferred so far, a block size in bytes, and the total size of the file.\n",
"> [...]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
2019-01-19 00:12:19 +00:00
"name": "stderr",
2019-01-18 23:55:05 +00:00
"output_type": "stream",
"text": [
2020-03-29 23:13:41 +00:00
"matryoshka.zip: 100%|██████████| 262k/262k [00:02<00:00, 87.9kB/s] \n"
2019-01-18 23:55:05 +00:00
]
}
],
2019-01-18 23:28:10 +00:00
"source": [
"import urllib, os\n",
2019-01-19 00:12:19 +00:00
"from tqdm import tqdm\n",
2021-02-10 20:23:35 +00:00
"urllib = getattr(urllib, 'request', urllib)\n",
2019-01-18 23:28:10 +00:00
"\n",
"class TqdmUpTo(tqdm):\n",
" \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
" def update_to(self, b=1, bsize=1, tsize=None):\n",
" \"\"\"\n",
" b : int, optional\n",
" Number of blocks transferred so far [default: 1].\n",
" bsize : int, optional\n",
" Size of each block (in tqdm units) [default: 1].\n",
" tsize : int, optional\n",
" Total size (in tqdm units). If [default: None] remains unchanged.\n",
" \"\"\"\n",
" if tsize is not None:\n",
" self.total = tsize\n",
2020-09-07 17:13:31 +00:00
" return self.update(b * bsize - self.n) # also sets self.n = b * bsize\n",
2019-01-18 23:28:10 +00:00
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
2020-05-02 23:04:23 +00:00
"with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n",
2019-01-18 23:28:10 +00:00
" desc=eg_link.split('/')[-1]) as t: # all optional kwargs\n",
" urllib.urlretrieve(eg_link, filename=os.devnull,\n",
2020-03-29 23:13:41 +00:00
" reporthook=t.update_to, data=None)\n",
" t.total = t.n"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
"Functional alternative in\n",
2019-01-19 00:12:19 +00:00
"[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"It is recommend to use `miniters=1` whenever there is potentially large\n",
"differences in iteration speed (e.g. downloading a file over a patchy\n",
"connection).\n",
"\n",
"**Wrapping read/write methods**\n",
"\n",
"To measure throughput through a file-like object's `read` or `write`\n",
"methods, use `CallbackIOWrapper`:\n",
"\n",
"```python\n",
"from tqdm import tqdm\n",
"from tqdm.utils import CallbackIOWrapper\n",
"\n",
"with tqdm(total=file_obj.size,\n",
" unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
" fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
" while True:\n",
" chunk = fobj.read(chunk_size)\n",
" if not chunk:\n",
" break\n",
" t.reset()\n",
" # ... continue to use `t` for something else\n",
"```\n",
"\n",
"Alternatively, use the even simpler `wrapattr` convenience function,\n",
"which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
"down to:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
2021-02-10 20:23:35 +00:00
"matryoshka.zip: 100%|██████████| 254k/254k [00:00<00:00, 602kB/s] \n"
2020-03-29 23:13:41 +00:00
]
}
],
"source": [
"import urllib, os\n",
"from tqdm import tqdm\n",
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
2021-02-10 20:23:35 +00:00
"response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n",
2020-03-29 23:13:41 +00:00
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
2021-02-10 20:23:35 +00:00
" miniters=1, desc=eg_link.split('/')[-1],\n",
" total=getattr(response, 'length', None)) as fout:\n",
" for chunk in response:\n",
2020-03-29 23:13:41 +00:00
" fout.write(chunk)"
2018-12-18 20:24:33 +00:00
]
},
2020-05-02 23:04:23 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
2021-02-10 20:23:35 +00:00
"The `requests` equivalent is nearly identical:"
2020-05-02 23:04:23 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"matryoshka.zip: 254kB [00:00, 460kB/s] \n"
]
}
],
"source": [
"import requests, os\n",
"from tqdm import tqdm\n",
"\n",
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
"response = requests.get(eg_link, stream=True)\n",
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
" miniters=1, desc=eg_link.split('/')[-1],\n",
2020-05-03 14:12:42 +00:00
" total=int(response.headers.get('content-length', 0))) as fout:\n",
2020-05-02 23:04:23 +00:00
" for chunk in response.iter_content(chunk_size=4096):\n",
" fout.write(chunk)"
]
},
2018-12-18 20:24:33 +00:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Pandas Integration\n",
"\n",
2019-01-18 23:28:10 +00:00
"Due to popular demand we've added support for `pandas` -- here's an example\n",
"for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:55:05 +00:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"my bar!: 100%|██████████| 6/6 [00:00<00:00, 193.96it/s]\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>0</th>\n",
" <th>1</th>\n",
" <th>2</th>\n",
" <th>3</th>\n",
" <th>4</th>\n",
" <th>5</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1296</td>\n",
" <td>1156</td>\n",
" <td>961</td>\n",
" <td>576</td>\n",
" <td>6889</td>\n",
" <td>7921</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>225</td>\n",
" <td>3844</td>\n",
" <td>8649</td>\n",
" <td>121</td>\n",
" <td>8281</td>\n",
" <td>4624</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3025</td>\n",
" <td>729</td>\n",
" <td>9025</td>\n",
" <td>4225</td>\n",
" <td>1024</td>\n",
" <td>8836</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>25</td>\n",
" <td>729</td>\n",
" <td>1936</td>\n",
" <td>6724</td>\n",
" <td>4900</td>\n",
" <td>3721</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>7225</td>\n",
" <td>1156</td>\n",
" <td>64</td>\n",
" <td>9604</td>\n",
" <td>441</td>\n",
" <td>961</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>5184</td>\n",
" <td>441</td>\n",
" <td>2500</td>\n",
" <td>5041</td>\n",
" <td>1936</td>\n",
" <td>5041</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>289</td>\n",
" <td>8281</td>\n",
" <td>8464</td>\n",
" <td>1089</td>\n",
" <td>7569</td>\n",
" <td>25</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>7225</td>\n",
" <td>1600</td>\n",
" <td>5625</td>\n",
" <td>5625</td>\n",
" <td>3364</td>\n",
" <td>7396</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>3025</td>\n",
" <td>4489</td>\n",
" <td>2704</td>\n",
" <td>400</td>\n",
" <td>6724</td>\n",
" <td>3364</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>6400</td>\n",
" <td>7396</td>\n",
" <td>4225</td>\n",
" <td>49</td>\n",
" <td>7921</td>\n",
" <td>4761</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>64</td>\n",
" <td>256</td>\n",
" <td>4356</td>\n",
" <td>8100</td>\n",
" <td>4761</td>\n",
" <td>4761</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>3600</td>\n",
" <td>729</td>\n",
" <td>4624</td>\n",
" <td>2500</td>\n",
" <td>25</td>\n",
" <td>441</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>5476</td>\n",
" <td>5329</td>\n",
" <td>6084</td>\n",
" <td>3025</td>\n",
" <td>3844</td>\n",
" <td>7396</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>16</td>\n",
" <td>6889</td>\n",
" <td>400</td>\n",
" <td>484</td>\n",
" <td>1225</td>\n",
" <td>169</td>\n",
" </tr>\n",
" <tr>\n",
" <th>14</th>\n",
" <td>3025</td>\n",
" <td>2209</td>\n",
" <td>9801</td>\n",
" <td>324</td>\n",
" <td>4900</td>\n",
" <td>4096</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>8649</td>\n",
" <td>6400</td>\n",
" <td>1936</td>\n",
" <td>3600</td>\n",
" <td>4900</td>\n",
" <td>2116</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>4356</td>\n",
" <td>3481</td>\n",
" <td>5476</td>\n",
" <td>7569</td>\n",
" <td>9216</td>\n",
" <td>16</td>\n",
" </tr>\n",
" <tr>\n",
" <th>17</th>\n",
" <td>961</td>\n",
" <td>196</td>\n",
" <td>4096</td>\n",
" <td>5476</td>\n",
" <td>8649</td>\n",
" <td>6724</td>\n",
" </tr>\n",
" <tr>\n",
" <th>18</th>\n",
" <td>1</td>\n",
" <td>4900</td>\n",
" <td>1156</td>\n",
" <td>2025</td>\n",
" <td>2916</td>\n",
" <td>1521</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19</th>\n",
" <td>6889</td>\n",
" <td>2809</td>\n",
" <td>9409</td>\n",
" <td>3249</td>\n",
" <td>7569</td>\n",
" <td>784</td>\n",
" </tr>\n",
" <tr>\n",
" <th>20</th>\n",
" <td>2116</td>\n",
" <td>49</td>\n",
" <td>225</td>\n",
" <td>81</td>\n",
" <td>8649</td>\n",
" <td>289</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>7569</td>\n",
" <td>2500</td>\n",
" <td>25</td>\n",
" <td>441</td>\n",
" <td>9025</td>\n",
" <td>6561</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>1</td>\n",
" <td>4489</td>\n",
" <td>3136</td>\n",
" <td>3025</td>\n",
" <td>3249</td>\n",
" <td>2116</td>\n",
" </tr>\n",
" <tr>\n",
" <th>23</th>\n",
" <td>5476</td>\n",
" <td>2116</td>\n",
" <td>9025</td>\n",
" <td>4096</td>\n",
" <td>1764</td>\n",
" <td>16</td>\n",
" </tr>\n",
" <tr>\n",
" <th>24</th>\n",
" <td>3025</td>\n",
" <td>961</td>\n",
" <td>3249</td>\n",
" <td>7056</td>\n",
" <td>3136</td>\n",
" <td>676</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25</th>\n",
" <td>2809</td>\n",
" <td>7744</td>\n",
" <td>1089</td>\n",
" <td>6724</td>\n",
" <td>0</td>\n",
" <td>961</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>225</td>\n",
" <td>400</td>\n",
" <td>7396</td>\n",
" <td>3721</td>\n",
" <td>784</td>\n",
" <td>7744</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>81</td>\n",
" <td>100</td>\n",
" <td>400</td>\n",
" <td>8836</td>\n",
" <td>2601</td>\n",
" <td>1600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>28</th>\n",
" <td>441</td>\n",
" <td>144</td>\n",
" <td>2704</td>\n",
" <td>1369</td>\n",
" <td>289</td>\n",
" <td>8281</td>\n",
" </tr>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>625</td>\n",
" <td>3025</td>\n",
" <td>8836</td>\n",
" <td>64</td>\n",
" <td>3025</td>\n",
" <td>9409</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99970</th>\n",
" <td>9216</td>\n",
" <td>8100</td>\n",
" <td>1369</td>\n",
" <td>529</td>\n",
" <td>625</td>\n",
" <td>7921</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99971</th>\n",
" <td>4</td>\n",
" <td>5184</td>\n",
" <td>3364</td>\n",
" <td>4356</td>\n",
" <td>5625</td>\n",
" <td>9604</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99972</th>\n",
" <td>484</td>\n",
" <td>6889</td>\n",
" <td>900</td>\n",
" <td>36</td>\n",
" <td>1444</td>\n",
" <td>3364</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99973</th>\n",
" <td>7744</td>\n",
" <td>3844</td>\n",
" <td>4489</td>\n",
" <td>5776</td>\n",
" <td>289</td>\n",
" <td>1225</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99974</th>\n",
" <td>1225</td>\n",
" <td>4489</td>\n",
" <td>9</td>\n",
" <td>2916</td>\n",
" <td>1369</td>\n",
" <td>100</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99975</th>\n",
" <td>1849</td>\n",
" <td>4356</td>\n",
" <td>169</td>\n",
" <td>2025</td>\n",
" <td>2704</td>\n",
" <td>7744</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99976</th>\n",
" <td>5929</td>\n",
" <td>7396</td>\n",
" <td>841</td>\n",
" <td>441</td>\n",
" <td>225</td>\n",
" <td>7056</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99977</th>\n",
" <td>676</td>\n",
" <td>7056</td>\n",
" <td>529</td>\n",
" <td>100</td>\n",
" <td>16</td>\n",
" <td>484</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99978</th>\n",
" <td>841</td>\n",
" <td>5476</td>\n",
" <td>8100</td>\n",
" <td>7569</td>\n",
" <td>6400</td>\n",
" <td>1156</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99979</th>\n",
" <td>4</td>\n",
" <td>7921</td>\n",
" <td>1225</td>\n",
" <td>5476</td>\n",
" <td>3844</td>\n",
" <td>7396</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99980</th>\n",
" <td>6241</td>\n",
" <td>1444</td>\n",
" <td>8281</td>\n",
" <td>1849</td>\n",
" <td>5625</td>\n",
" <td>3721</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99981</th>\n",
" <td>2401</td>\n",
" <td>121</td>\n",
" <td>169</td>\n",
" <td>49</td>\n",
" <td>2916</td>\n",
" <td>1296</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99982</th>\n",
" <td>1</td>\n",
" <td>1764</td>\n",
" <td>441</td>\n",
" <td>8836</td>\n",
" <td>6561</td>\n",
" <td>169</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99983</th>\n",
" <td>1444</td>\n",
" <td>3600</td>\n",
" <td>9025</td>\n",
" <td>8281</td>\n",
" <td>7569</td>\n",
" <td>1444</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99984</th>\n",
" <td>5776</td>\n",
" <td>1600</td>\n",
" <td>9</td>\n",
" <td>9409</td>\n",
" <td>961</td>\n",
" <td>3249</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99985</th>\n",
" <td>2916</td>\n",
" <td>4356</td>\n",
" <td>64</td>\n",
" <td>1681</td>\n",
" <td>49</td>\n",
" <td>1936</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99986</th>\n",
" <td>5929</td>\n",
" <td>1089</td>\n",
" <td>169</td>\n",
" <td>4225</td>\n",
" <td>4225</td>\n",
" <td>1296</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99987</th>\n",
" <td>3481</td>\n",
" <td>2116</td>\n",
" <td>5041</td>\n",
" <td>7569</td>\n",
" <td>6724</td>\n",
" <td>25</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99988</th>\n",
" <td>3136</td>\n",
" <td>4225</td>\n",
" <td>676</td>\n",
" <td>484</td>\n",
" <td>961</td>\n",
" <td>4624</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99989</th>\n",
" <td>2500</td>\n",
" <td>7056</td>\n",
" <td>900</td>\n",
" <td>4096</td>\n",
" <td>3844</td>\n",
" <td>8100</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99990</th>\n",
" <td>784</td>\n",
" <td>900</td>\n",
" <td>2916</td>\n",
" <td>4356</td>\n",
" <td>7396</td>\n",
" <td>2916</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99991</th>\n",
" <td>3721</td>\n",
" <td>1764</td>\n",
" <td>16</td>\n",
" <td>9</td>\n",
" <td>5929</td>\n",
" <td>4900</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99992</th>\n",
" <td>64</td>\n",
" <td>2809</td>\n",
" <td>3025</td>\n",
" <td>9</td>\n",
" <td>5625</td>\n",
" <td>484</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99993</th>\n",
" <td>2116</td>\n",
" <td>1600</td>\n",
" <td>5929</td>\n",
" <td>81</td>\n",
" <td>625</td>\n",
" <td>441</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99994</th>\n",
" <td>1521</td>\n",
" <td>529</td>\n",
" <td>2025</td>\n",
" <td>2809</td>\n",
" <td>3364</td>\n",
" <td>5929</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99995</th>\n",
" <td>576</td>\n",
" <td>1225</td>\n",
" <td>1764</td>\n",
" <td>3721</td>\n",
" <td>0</td>\n",
" <td>144</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99996</th>\n",
" <td>625</td>\n",
" <td>5476</td>\n",
" <td>3364</td>\n",
" <td>9409</td>\n",
" <td>100</td>\n",
" <td>4900</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99997</th>\n",
" <td>900</td>\n",
" <td>1444</td>\n",
" <td>1521</td>\n",
" <td>676</td>\n",
" <td>49</td>\n",
" <td>7056</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99998</th>\n",
" <td>6084</td>\n",
" <td>36</td>\n",
" <td>289</td>\n",
" <td>6561</td>\n",
" <td>7921</td>\n",
" <td>256</td>\n",
" </tr>\n",
" <tr>\n",
" <th>99999</th>\n",
" <td>1089</td>\n",
" <td>784</td>\n",
" <td>3969</td>\n",
" <td>2500</td>\n",
" <td>2401</td>\n",
" <td>2500</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>100000 rows ×
"</div>"
],
"text/plain": [
" 0 1 2 3 4 5\n",
"0 1296 1156 961 576 6889 7921\n",
"1 225 3844 8649 121 8281 4624\n",
"2 3025 729 9025 4225 1024 8836\n",
"3 25 729 1936 6724 4900 3721\n",
"4 7225 1156 64 9604 441 961\n",
"5 5184 441 2500 5041 1936 5041\n",
"6 289 8281 8464 1089 7569 25\n",
"7 7225 1600 5625 5625 3364 7396\n",
"8 3025 4489 2704 400 6724 3364\n",
"9 6400 7396 4225 49 7921 4761\n",
"10 64 256 4356 8100 4761 4761\n",
"11 3600 729 4624 2500 25 441\n",
"12 5476 5329 6084 3025 3844 7396\n",
"13 16 6889 400 484 1225 169\n",
"14 3025 2209 9801 324 4900 4096\n",
"15 8649 6400 1936 3600 4900 2116\n",
"16 4356 3481 5476 7569 9216 16\n",
"17 961 196 4096 5476 8649 6724\n",
"18 1 4900 1156 2025 2916 1521\n",
"19 6889 2809 9409 3249 7569 784\n",
"20 2116 49 225 81 8649 289\n",
"21 7569 2500 25 441 9025 6561\n",
"22 1 4489 3136 3025 3249 2116\n",
"23 5476 2116 9025 4096 1764 16\n",
"24 3025 961 3249 7056 3136 676\n",
"25 2809 7744 1089 6724 0 961\n",
"26 225 400 7396 3721 784 7744\n",
"27 81 100 400 8836 2601 1600\n",
"28 441 144 2704 1369 289 8281\n",
"29 625 3025 8836 64 3025 9409\n",
"... ... ... ... ... ... ...\n",
"99970 9216 8100 1369 529 625 7921\n",
"99971 4 5184 3364 4356 5625 9604\n",
"99972 484 6889 900 36 1444 3364\n",
"99973 7744 3844 4489 5776 289 1225\n",
"99974 1225 4489 9 2916 1369 100\n",
"99975 1849 4356 169 2025 2704 7744\n",
"99976 5929 7396 841 441 225 7056\n",
"99977 676 7056 529 100 16 484\n",
"99978 841 5476 8100 7569 6400 1156\n",
"99979 4 7921 1225 5476 3844 7396\n",
"99980 6241 1444 8281 1849 5625 3721\n",
"99981 2401 121 169 49 2916 1296\n",
"99982 1 1764 441 8836 6561 169\n",
"99983 1444 3600 9025 8281 7569 1444\n",
"99984 5776 1600 9 9409 961 3249\n",
"99985 2916 4356 64 1681 49 1936\n",
"99986 5929 1089 169 4225 4225 1296\n",
"99987 3481 2116 5041 7569 6724 25\n",
"99988 3136 4225 676 484 961 4624\n",
"99989 2500 7056 900 4096 3844 8100\n",
"99990 784 900 2916 4356 7396 2916\n",
"99991 3721 1764 16 9 5929 4900\n",
"99992 64 2809 3025 9 5625 484\n",
"99993 2116 1600 5929 81 625 441\n",
"99994 1521 529 2025 2809 3364 5929\n",
"99995 576 1225 1764 3721 0 144\n",
"99996 625 5476 3364 9409 100 4900\n",
"99997 900 1444 1521 676 49 7056\n",
"99998 6084 36 289 6561 7921 256\n",
"99999 1089 784 3969 2500 2401 2500\n",
"\n",
"[100000 rows x 6 columns]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
2018-12-18 20:24:33 +00:00
"source": [
"import pandas as pd\n",
"import numpy as np\n",
2019-01-18 23:55:05 +00:00
"from tqdm import tqdm\n",
2018-12-18 20:24:33 +00:00
"\n",
2019-01-18 23:28:10 +00:00
"df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n",
2018-12-18 20:24:33 +00:00
"\n",
"# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n",
2020-03-29 23:13:41 +00:00
"# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n",
2018-12-18 20:24:33 +00:00
"tqdm.pandas(desc=\"my bar!\")\n",
"\n",
"# Now you can use `progress_apply` instead of `apply`\n",
"# and `progress_map` instead of `map`\n",
"df.progress_apply(lambda x: x**2)\n",
"# can also groupby:\n",
"# df.groupby(0).progress_apply(lambda x: x**2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"In case you're interested in how this works (and how to modify it for\n",
"your own callbacks), see the\n",
"[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n",
"import the module and run `help()`.\n",
"\n",
"### Keras Integration\n",
"\n",
"```python\n",
"A `keras` callback is also available:\n",
"\n",
"from tqdm.keras import TqdmCallback\n",
"\n",
"...\n",
"\n",
"model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n",
"```\n",
2019-01-18 23:28:10 +00:00
"\n",
"### IPython/Jupyter Integration\n",
"\n",
2020-03-29 23:13:41 +00:00
"IPython/Jupyter is supported via the `tqdm.notebook` submodule:"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
2020-03-29 23:13:41 +00:00
"from tqdm.notebook import trange, tqdm\n",
2019-01-18 23:28:10 +00:00
"from time import sleep\n",
"\n",
2020-03-29 23:13:41 +00:00
"for i in trange(3, desc='1st loop'):\n",
" for j in tqdm(range(100), desc='2nd loop'):\n",
2019-01-18 23:28:10 +00:00
" sleep(0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In addition to `tqdm` features, the submodule provides a native Jupyter\n",
2020-03-29 23:13:41 +00:00
"widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n",
"bars and colour hints (blue: normal, green: completed, red:\n",
"error/interrupt, light blue: no ETA); as demonstrated below.\n",
2019-01-18 23:28:10 +00:00
"\n",
2021-01-12 13:58:24 +00:00
"\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"The `notebook` version supports percentage or pixels for overall width\n",
"(e.g.: `ncols='100%'` or `ncols='480px'`).\n",
"\n",
"It is also possible to let `tqdm` automatically choose between console\n",
"or notebook versions by using the `autonotebook` submodule:"
2019-01-18 23:28:10 +00:00
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tqdm.autonotebook import tqdm\n",
"tqdm.pandas()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2020-03-29 23:13:41 +00:00
"Note that this will issue a `TqdmExperimentalWarning` if run in a\n",
"notebook since it is not meant to be possible to distinguish between\n",
"`jupyter notebook` and `jupyter console`. Use `auto` instead of\n",
"`autonotebook` to suppress this warning.\n",
2019-01-18 23:28:10 +00:00
"\n",
2020-03-29 23:13:41 +00:00
"Note that notebooks will display the bar in the cell where it was\n",
"created. This may be a different cell from the one where it is used. If\n",
"this is not desired, the creation of the bar must be delayed/moved to\n",
"the cell where it is desired to be displayed.\n",
"\n",
"Another possibility is to have a single bar (near the top of the\n",
"notebook) which is constantly re-used (using `reset()` rather than\n",
"`close()`). For this reason, the notebook version (unlike the CLI\n",
"version) does not automatically call `close()` upon `Exception`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tqdm.notebook import tqdm\n",
"pbar = tqdm()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# different cell\n",
"iterable = range(100)\n",
"pbar.reset(total=len(iterable)) # initialise with new `total`\n",
"for i in iterable:\n",
" pbar.update()\n",
"pbar.refresh() # force print final status but don't `close()`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-12-18 20:24:33 +00:00
"### Writing messages\n",
"\n",
2020-03-29 23:13:41 +00:00
"This is a work in progress (see\n",
"[#737](https://github.com/tqdm/tqdm/issues/737)).\n",
"\n",
2019-01-18 23:28:10 +00:00
"Since `tqdm` uses a simple printing mechanism to display progress bars,\n",
2020-03-29 23:13:41 +00:00
"you should not write any message in the terminal using `print()` while a\n",
"progressbar is open.\n",
2018-12-18 20:24:33 +00:00
"\n",
2019-01-18 23:28:10 +00:00
"To write messages in the terminal without any collision with `tqdm` bar\n",
"display, a `.write()` method is provided:"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "code",
2019-01-18 23:28:10 +00:00
"execution_count": null,
2018-12-18 20:24:33 +00:00
"metadata": {},
2019-01-18 23:28:10 +00:00
"outputs": [],
2018-12-18 20:24:33 +00:00
"source": [
2020-03-29 23:13:41 +00:00
"from tqdm.auto import tqdm, trange\n",
2018-12-18 20:24:33 +00:00
"from time import sleep\n",
"\n",
"bar = trange(10)\n",
"for i in bar:\n",
" # Print using tqdm class method .write()\n",
" sleep(0.1)\n",
" if not (i % 3):\n",
" tqdm.write(\"Done task %i\" % i)\n",
" # Can also use bar.write()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2019-01-18 23:28:10 +00:00
"By default, this will print to standard output `sys.stdout`. but you can\n",
2020-03-29 23:13:41 +00:00
"specify any file-like object using the `file` argument. For example,\n",
"this can be used to redirect the messages writing to a log file or class.\n",
"\n",
"[](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://caspersci.uk.to/images/tqdm.png&f=https://raw.githubusercontent.com/tqdm/tqdm/master/images/logo.gif&style=social)|(Since 19 May 2016)\n",
"-|-"
2018-12-18 20:24:33 +00:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Do your own experiments here 👇\n",
"\n",
"Try `tqdm` youself by adding your code below and running your own experiments."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import tqdm\n",
"\n",
"# your code here\n",
"tqdm."
]
}
],
"metadata": {
"kernelspec": {
2021-02-10 20:24:48 +00:00
"display_name": "Python 3",
2018-12-18 20:24:33 +00:00
"language": "python",
2021-02-10 20:24:48 +00:00
"name": "python3"
2018-12-18 20:24:33 +00:00
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
2021-02-10 20:24:48 +00:00
"version": 3
2018-12-18 20:24:33 +00:00
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
2021-02-10 20:24:48 +00:00
"pygments_lexer": "ipython3",
"version": "3.7.8"
2018-12-18 20:24:33 +00:00
}
},
"nbformat": 4,
"nbformat_minor": 2
}
