Table of Contents
Kivy garden
When we originally developed the garden concept, where people can host their widgets (flowers) that other people can download, we also developed the kivy-garden tool that lets one install these flowers. The structure of these flowers were specified such that the garden tool can simply download the files and then kivy, through a import hook, would be able to use these flowers. This meant we wouldn't use the typical python packaging mechanism (setup.py, proper package structure).
However, there are many issues with this approach that we discovered over the years, such as no cython support (or anything that requires setup), having to teach users how to use a new custom tool, and maintenance issues where we don't have the resources to deal with bugs that come up. Additionally, by default installing packages outside typical python hierarchies (in the user home folder), we circumvent any virtual environment or environment localization. Finally, there's no easy way to specify a complete installation with a requirements.txt type file.
Proposal
I have proposed a new approach to garden that uses normal python packaging with pip and have provided a detailed migration path and some demo packages to make the migration easier. Specifically, here's how garden packages would appear:
- We will have a proper kivy_garden namespace where garden packages will be installed to. So e.g. if you want to install the graph flower, you'd do
pip install kivy_garden.graph
and it would install the package underkivy_garden/graph
. This would be external to kivy and part of the python environment like any normal python package. - Garden packages would have a normal setup.py and complete python package structure. I wrote a demo pure-python and cython-python flower that should be used for new packages and for migrating old packages.
- There's a migration guide here
- Demo converting pure-python graph from https://github.com/kivy-garden/garden.graph to https://github.com/kivy-garden/graph
- Demo converting cython-python collider from https://github.com/kivy-garden/garden.collider to https://github.com/kivy-garden/collider
- Instead of the garden tool, users will have a choice to either
- install directly from pypi, e.g.
pip install kivy_garden.graph
.- kivy will not make automated releases to pypi, it would be up to contributors to do that. But we do have guides for that in the flower demo.
- Install directly from github
pip install https://github.com/kivy-garden/graph/archive/master.zip
orpip install https://github.com/kivy-garden/graph/archive/some_release.zip
. - Install using kivy's simple pypi server that we host on github:
pip install kivy_garden.graph --extra-index-url https://kivy-garden.github.io/simple/
. This server is hosted on github. To add new flowers people will be making PRs. Similarly, to release new versions, add pre-compiled wheels stored on github release assets, users will be making PRs to that repo. Eventually we can automate this, but for now it's easy to make PRs to add new packages or new releases to existing packages.
- install directly from pypi, e.g.
- Packaging with pyinstaller will work as with normal python packages.
The legacy garden packages and the garden tool will be able to be installed side by side with the newer packages because they are installed to different places.
Kivy deps
We invented the kivy.deps.xxx
packages to be able to ship kivy in wheel form. We needed a way to ship the dependencies so we decided to store the deps under kivy itself in kivy/deps/xxx
because that seemed the best option at the time.
However, as it turns out this has some issues that currently cannot be overcome without manual copying of files. Specifically, if kivy is installed as editable and we install a dependency, pip is unable to install it at the editable location and instead installs it under site-packages. This breaks kivy and the dependency has to be manually copied to the editable location.
Also, installing dependencies directly under kivy is not a great choice from an isolation POV. And by doing this we are not using the typical form of namepsace packages.
Instead, like kivy_garden
, I am proposing that we add support for the kivy_deps
name spaces with packages such as kivy_deps.sdl2
. These packages would be installed in kivy_deps/xxx
, outside the kivy hierarchy. Here's the PR for this proposal.
Migration process
pip does not distinguish between packages named kivy.deps.xxx
and kivy_deps.xxx
because to pip, both are converted to kivy-deps-xxx
. What this means is that pip and pypi consider kivy.deps.xxx
and kivy_deps.xxx
as the same project and if you upload them they are stored in the same project on pypi. Similarly, no matter which you request with pip install ...
pip will choose the one with the latest version specifier whether it is kivy.deps.xxx
or kivy_deps.xxx
. This is one of the reasons why we should move away from the kivy/deps/xxx
hierarchy as that leads to more conflicts.
Even though pip does not differentiate between kivy.deps.xxx
and kivy_deps.xxx
, they still are installed to the proper locations (kivy.deps.xxx
to kivy/deps/xxx
and kivy_deps.xxx
to kivy_deps/xxx
).
So I propose that we release all the kivy_deps.xxx
packages such that their version is newer than the kivy.deps.xxx
packages and that we deprecate the kivy.deps.xxx
packages so that their version will not increase. This is easily done. What this is means is as follows:
- If you are on release 1.11.0 (assuming this is merged before that release) or kivy master, you should just install packages with
kivy_deps.xxx
, this will install them tokivy_deps/xxx
and everything will work. - If you are on a older kivy and don't plan to upgrade, you should pin all your
kivy.deps.xxx
to specific versions that were released at the time of your kivy release. This will prevent the newerkivy_deps.xxx
from being installed. You will need to do this anyway because the latest gstreamer may cause issues for older kivy version before 1.11.0. And we should get our users in the habit of pinning their versions anyway. - If you are migrating to 1.11.0, you should uninstall all your deps with pip, or make sure they are upgraded with pip (
pip install --upgrade kivy_deps...
) when updating to 1.110 or newer. This will give you thekivy_deps/xxx
versions.