mirror of https://github.com/tqdm/tqdm.git
170 lines
6.4 KiB
Markdown
170 lines
6.4 KiB
Markdown
---
|
||
title: '`tqdm`: A Fast, Extensible Progress Meter for Python and CLI'
|
||
tags:
|
||
- progressbar
|
||
- progressmeter
|
||
- progress-bar
|
||
- meter
|
||
- rate
|
||
- eta
|
||
- console
|
||
- terminal
|
||
- time
|
||
- progress
|
||
- bar
|
||
- gui
|
||
- python
|
||
- parallel
|
||
- cli
|
||
- utilities
|
||
- shell
|
||
- batch
|
||
authors:
|
||
- name: Casper O da Costa-Luis
|
||
orcid: 0000-0002-7211-1557
|
||
affiliation: 1
|
||
affiliations:
|
||
- name: "Independent (Non-affiliated)"
|
||
index: 1
|
||
date: 16 February 2019
|
||
bibliography: paper.bib
|
||
---
|
||
|
||
# Introduction
|
||
|
||
**`tqdm`** is a progress bar library designed to be fast and extensible. It is
|
||
written in Python, though ports in other languages are available. `tqdm` means
|
||
**progress** in Arabic (*taqadum* [@tqdm-ar]) and is an abbreviation for
|
||
**I love you so much** in Spanish (*te quiero demasiado* [@tqdm-es]).
|
||
|
||
It is a common programming problem to have iterative operations where progress
|
||
monitoring is desirable or advantageous. Including statements within a `for` loop to `print` out the current iteration number is a common strategy. However, there are many improvements which could be made in such a scenario:
|
||
|
||
- preventing excessive printing, such as only displaying every $n$^th^
|
||
iteration;
|
||
- displaying iteration rate;
|
||
- displaying elapsed and estimated completion times, and
|
||
- showing all of the above on one continuously updating line.
|
||
|
||
Addressing all these issues may well take up more developer time and effort than
|
||
the rest of the content of the loop. Any changes to iteration rates or attempts
|
||
to re-use the printing logic in a different loop may well result in suboptimal
|
||
display rates -- displaying every $n$^th^ iteration may be too (in)frequent --
|
||
requiring manual adjustment of $n$ to fix.
|
||
|
||
`tqdm` addresses all of these problems once and for all, taking advantage of
|
||
Pythonic patterns to make it a trivial task to add visually appealing,
|
||
customisable progress bars without any significant performance degradation even
|
||
in the most demanding of scenarios.
|
||
|
||
`tqdm` is intended to be used in frontends (giving end users a visual indication
|
||
of progress of computations or data transfer). It is also useful for developers
|
||
for debugging purposes, both as a profiling tool and also as a way of displaying
|
||
logging information of an iterative task (such as error during training of
|
||
machine learning algorithms). Due to its ease of use, the library is also an
|
||
ideal candidate for inclusion in Python educational courses. For general (not
|
||
necessarily Python) purposes, the command-line interface (CLI) mode further
|
||
presents a useful tool for CLI users and system administrators monitoring data
|
||
flow through pipes.
|
||
|
||
# Features
|
||
|
||
Exhaustive documentation may be found on the project's [home
|
||
page](https://github.com/tqdm/tqdm/#documentation).
|
||
|
||
The two basic use cases are within Python code and within a CLI:
|
||
|
||
## Python Iterable Wrapper
|
||
|
||
`tqdm`'s primary (and original) use is as a wrapper around Python iterables. A
|
||
simple case would be:
|
||
|
||
```python
|
||
from tqdm import tqdm
|
||
from time import sleep
|
||
for i in tqdm(range(100)):
|
||
sleep(0.1)
|
||
100%|#########################################| 100/100 [00:10<00:00, 9.95it/s]
|
||
```
|
||
|
||
Supported features include:
|
||
|
||
- Display customisation via arguments such as `desc`, `postfix` and `bar_format`
|
||
- Automatic limiting of display updates to avoid slowing down due to excessive
|
||
iteration rates [@stdout]
|
||
- Automatic detection of console width to fill the display
|
||
- Automatic use of Unicode to render smooth-filling progress bars on supported
|
||
terminals
|
||
- Support for custom rendering frontends, including:
|
||
* Command-line interface
|
||
* *Jupyter* HTML notebooks
|
||
* `matplotlib`
|
||
- Support for custom hooks/callbacks, including:
|
||
* `pandas`
|
||
* `keras` [@keras]
|
||
|
||
## Command-line Interface (CLI)
|
||
|
||
A CLI is also provided, where `tqdm` may be used a pipe:
|
||
|
||
```sh
|
||
# count lines of text in all *.txt files
|
||
$ cat *.txt | wc -l
|
||
1075075
|
||
# same but with continuously updating progress information
|
||
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale | wc -l
|
||
1.08Mloc [00:07, 142kloc/s]
|
||
# same if `total` is known
|
||
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale --total 1075075 | wc -l
|
||
100%|#####################################| 1.08/1.08M [00:07<00:00, 142kloc/s]
|
||
1075075
|
||
```
|
||
|
||
# Availability
|
||
|
||
The package supports both Python versions 2 and 3, and is available for download
|
||
via `conda` [@conda], `pip` [@pypi], `snap` [@snapcraft], `docker` [@docker],
|
||
and *Zenodo* [@zenodo].
|
||
Web-based Jupyter interactive demonstrations are also available
|
||
[@notebooks;@binder]
|
||
|
||
Unit tests are run at least weekly on cloud-based continuous integration
|
||
[@travis], with code style and security issues checked on
|
||
[Codacy](https://app.codacy.com/project/tqdm/tqdm/dashboard) [@code-review].
|
||
Coverage is reported on [Coveralls](https://coveralls.io/github/tqdm/tqdm) and
|
||
[Codecov](https://codecov.io/gh/tqdm/tqdm), and performance is monitored against
|
||
regression [@asv].
|
||
|
||
# Impact
|
||
|
||
As of January 2019, `tqdm` has accumulated over 20 million downloads
|
||
[@pypi-downloads], and 315 thousand code inclusions [@tqdm-results]. Dependants
|
||
of `tqdm` include 23 thousand repositories [@tqdm-dependents] and 7 thousand
|
||
libraries [@lib-io]. `tqdm` has a SourceRank of 22 [@sourcerank], placing it in
|
||
the world's top 20 Python packages as of early 2019 [@sourcerank-descending].
|
||
|
||
The source code of `tqdm` is hosted on GitHub, where it has received over 9
|
||
thousand stars [@stars;@stars-hist], and was top trending repository during a
|
||
period in December 2015 [@trend-hist]. The documentation has received over 500
|
||
thousand hits [@hits], with highest rates during weekdays. Historical reading
|
||
rates have also trended upwards at the end of holiday periods. This implies
|
||
widespread use in commercial and academic settings.
|
||
[OpenHub](https://www.openhub.net/p/tqdm) valuates the work according to the
|
||
constructive cost model (COCOMO) as being worth approximately $50,000.
|
||
|
||
The library has also been used in several textbooks [@miller;@boxel;@nandy] and
|
||
peer-reviewed scientific publications
|
||
[@stein;@cook;@madhikar;@palmer;@knight;@moriwaki;@jackson].
|
||
The [`tqdm` wiki](https://github.com/tqdm/tqdm/wiki) also lists other references
|
||
in public media.
|
||
|
||
# Licence
|
||
|
||
`tqdm`'s source code is OSS, and all versions are archived at the DOI
|
||
[10.5281/zenodo.595120](https://doi.org/10.5281/zenodo.595120). The primary
|
||
maintainer [Casper da Costa-Luis](https://github.com/casperdcl) releases
|
||
contributions under the terms of the MPLv2.0, while all other contributions are
|
||
released under the terms of the MIT licence [@licence].
|
||
|
||
# References
|