12 KiB

Raw Blame History

HOW TO CONTRIBUTE TO TQDM

TL;DR: Skip to [QUICK DEV SUMMARY]

This file describes how to

contribute changes to the project, and
upload released to the PyPI repository.

Most of the management commands have been directly placed inside the Makefile:

make [<alias>]  # on UNIX-like environments
python -m pymake [<alias>]  # if make is unavailable

The latter depends on py-make>=0.1.0.

Use the alias help (or leave blank) to list all available aliases.

HOW TO COMMIT CONTRIBUTIONS

Contributions to the project are made using the "Fork & Pull" model. The typical steps would be:

create an account on github
fork tqdm
make a local clone: git clone https://github.com/your_account/tqdm.git
make changes on the local copy
test (see below) and commit changes git commit -a -m "my message"
push to your GitHub account: git push origin
create a Pull Request (PR) from your GitHub fork (go to your fork's webpage and click on "Pull Request." You can then add a message to describe your proposal.)

WHAT CODE LAYOUT SHOULD I FOLLOW?

Don't worry too much - maintainers can help reorganise contributions. However it would be helpful to bear in mind:

The standard core of tqdm, i.e. tqdm.std.tqdm
- must have no dependencies apart from pure python built-in standard libraries
- must have negligible impact on performance
- should have 100% coverage by unit tests
- should be appropriately commented
- should have well-formatted docstrings for functions
  - under 76 chars (incl. initial spaces) to avoid linebreaks in terminal pagers
  - use two spaces between variable name and colon, specify a type, and most likely state that it's optional: VAR<space><space>:<space>TYPE[, optional]
  - use [default: ...] for default values of keyword arguments
- will not break backward compatibility unless there is a very good reason
  - e.g. breaking py26 compatibility purely in favour of minor readability changes (such as converting dict(a=1) to {'a': 1}) is not a good enough reason
- API changes should be discussed carefully
- remember, with millions of downloads per month, tqdm must be extremely fast and reliable
Any other kind of change may be included in a (possibly new) submodule
- submodules are likely single python files under the main tqdm/ directory
- submodules extending tqdm.std.tqdm or any other module (e.g. tqdm.notebook.tqdm, tqdm.gui.tqdm)
- CLI wrapper tqdm.cli
  - if a newly added tqdm.std.tqdm option is not supported by the CLI, append to tqdm.cli.UNSUPPORTED_OPTS
- can implement anything from experimental new features to support for third-party libraries such as pandas, numpy, etc.
- submodule maturity
  - alpha: experimental; missing unit tests, comments, and/or feedback; raises tqdm.TqdmExperimentalWarning
  - beta: well-used; commented, perhaps still missing tests
  - stable: >10 users; commented, 80% coverage
.meta/
- A "hidden" folder containing helper utilities not strictly part of the tqdm distribution itself

TESTING

Once again, don't worry too much - tests are automated online, and maintainers can also help.

To test functionality (such as before submitting a Pull Request), there are a number of unit tests.

Standard unit tests

The standard way to run the tests:

install tox
cd to the root of the tqdm directory (in the same folder as this file)
run the following command:

[python -m py]make test
# or:
tox --skip-missing-interpreters

This will build the module and run the tests in a virtual environment. Errors and coverage rates will be output to the console/log. (Ignore missing interpreters errors - these are due to the local machine missing certain versions of Python.)

Note: to install all versions of the Python interpreter that are specified in tox.ini, you can use MiniConda to install a minimal setup. You must also ensure that each distribution has an alias to call the Python interpreter (e.g. python313 for Python 3.13's interpreter).

Alternative unit tests with pytest

Alternatively, use pytest to run the tests just for the current Python version:

install test requirements: [python -m py]make install_test
run the following command:

[python -m py]make alltests

MANAGE A NEW RELEASE

This section is intended for the project's maintainers and describes how to build and upload a new release. Once again, [python -m py]make [<alias>] will help. Also consider pip installing development utilities: [python -m py]make install_build at a minimum, or a more thorough conda env create.

Pre-commit Hook

It's probably a good idea to use the pre-commit (pip install pre-commit) helper.

Run pre-commit install for convenient local sanity-checking.

Semantic Versioning

The tqdm repository managers should:

follow the Semantic Versioning convention for tagging

Checking `pyproject.toml`

To check that the pyproject.toml file is compliant with PyPI requirements (e.g. version number; reStructuredText in README.rst) use:

[python -m py]make testsetup

To upload just metadata (including overwriting mistakenly uploaded metadata) to PyPI, use:

[python -m py]make pypimeta

Merging Pull Requests

This section describes how to cleanly merge PRs.

1 Rebase

From your project repository, merge and test (replace pr-branch-name as appropriate):

git fetch origin
git checkout -b pr-branch-name origin/pr-branch-name
git rebase master

If there are conflicts:

git mergetool
git rebase --continue

2 Push

Update branch with the rebased history:

git push origin pr-branch-name --force

Non maintainers can stop here.

Note: NEVER just git push --force (this will push all local branches, overwriting remotes).

3 Merge

git checkout master
git merge --no-ff pr-branch-name

4 Test

[python -m py]make alltests

5 Push to master

git push origin master

Building a Release and Uploading to PyPI

Formally publishing requires additional steps: testing and tagging.

Test

Ensure that all online CI tests have passed.

Upload

GitHub Actions (GHA) CI should automatically do this after pushing tags. Manual instructions are given below in case of failure.

Build tqdm into a distributable python package:

[python -m py]make build

This will generate several builds in the dist/ folder. On non-windows machines the windows exe installer may fail to build. This is normal.

Finally, upload everything to PyPI. This can be done easily using the twine module:

[python -m py]make pypi

Also, the new release can (should) be added to GitHub by creating a new release from the web interface; uploading packages from the dist/ folder created by [python -m py]make build. The wiki can be automatically updated with GitHub release notes by running make within the wiki repository.

Docker images may be uploaded to https://hub.docker.com/r/tqdm/tqdm. Assuming docker is installed:

make -B docker
docker login
docker push tqdm/tqdm:latest
docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)

Snaps may be uploaded to https://snapcraft.io/tqdm. Assuming snapcraft is installed (snap install snapcraft --classic --beta):

make snap
snapcraft login
snapcraft push tqdm*.snap --release stable

Notes

you can also test on the PyPI test servers test.pypi.org before the real deployment
in case of a mistake, you can delete an uploaded release on PyPI, but you cannot re-upload another with the same version number
in case of a mistake in the metadata on PyPI (e.g. bad README), updating just the metadata is possible: [python -m py]make pypimeta

Updating Websites

The most important file is .readme.rst, which should always be kept up-to-date and in sync with the in-line source documentation. This will affect all of the following:

README.rst (generated by mkdocs.py during make build)
The main repository site which automatically serves the latest README.rst as well as links to all of GitHub's features. This is the preferred online referral link for tqdm.
The PyPI mirror which automatically serves the latest release built from README.rst as well as links to past releases.
Many external web crawlers.

Additionally (less maintained), there exists:

A wiki which is publicly editable.
The gh-pages project which is built from the gh-pages branch, which is built using asv.
The gh-pages root which is built from a separate github.io repo.

Helper Bots

There are some helpers in .github/workflows to assist with maintenance.

Comment Bot
- allows maintainers to write /tag vM.m.p commit_hash in an issue/PR to create a tag
Post Release
- automatically updates the wiki
- automatically updates the gh-pages root
Benchmark
- automatically updates the gh-pages project

QUICK DEV SUMMARY

For experienced devs, once happy with local master, follow the steps below. Much is automated so really it's steps 1-5, then 11(a).

test ([python -m py]make alltests or rely on pre-commit)
git commit [--amend] # -m "bump version"
git push
wait for tests to pass a) in case of failure, fix and go back to (1)
git tag vM.m.p && git push --tags or comment /tag vM.m.p commit_hash
[AUTO:GHA] [python -m py]make distclean
[AUTO:GHA] [python -m py]make build
[AUTO:GHA] upload to PyPI. either: a) [python -m py]make pypi, or b) twine upload -s -i $(git config user.signingkey) dist/tqdm-*
[AUTO:GHA] upload to docker hub: a) make -B docker b) docker push tqdm/tqdm:latest c) docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)
[AUTO:GHA] upload to snapcraft: a) make snap, and b) snapcraft push tqdm*.snap --release stable
Wait for GHA to draft a new release on https://github.com/tqdm/tqdm/releases a) replace the commit history with helpful release notes, and click publish b) [AUTO:GHA] attach dist/tqdm-* binaries (usually only *.whl*)
[SUB][AUTO:GHA-rel] run make in the wiki submodule to update release notes
[SUB][AUTO:GHA-rel] run make deploy in the docs submodule to update website
[SUB][AUTO:GHA-rel] accept the automated PR in the feedstock submodule to update conda
[AUTO:GHA-rel] update the gh-pages project benchmarks a) [python -m py]make testasvfull b) asv gh-pages

Key:

[AUTO:GHA]: GitHub Actions CI should automatically do this after git push --tags (5)
[AUTO:GHA-rel]: GitHub Actions CI should automatically do this after release (11a)
[SUB]: Requires one-time make submodules to clone docs, wiki, and feedstock

12 KiB Raw Blame History