In the first part of this mini-series I would like to explain how an user can:
- install a Python package,
- pin version of dependency,
- save dependencies for later.
You can also browse other posts from this mini-series:
- Part 1: Installation (current)
- Part 2: Venvs
Python’s default package manager is called pip which replaced a previous one,
easy_install. Basic usage of pip is simple:
pip install this,
uninstall that is most of what you will do on a daily routine. But don’t be
disappointed - pip’s commands are flexible thanks to a number of switches and
One of the most common switches is, from my experience,
-r for shortcut). It allows an user to install packages referenced in a
requirements file. Pip allows
-r option to be used multiple times so
there might (and probably - should) be distinct files for running a project and
developing (or testing) it. Pip will install wheel releases, if available,
by default. Wheels are Python’s prebuild packages, thanks to them you don’t
need to compile a plugin written in C on your dev machine or a server. To learn
more about wheels go to packaging.python.org.
Version pinning and requirements file
In the previous section I’ve mentioned requirements file. How to prepare one and what is its format?
It is just a regular text file, usually with a
.txt extension, which list
packages names you want to install. For instance such a requirements file:
will tell pip to install the latest versions of aiohttp and aiohttp_jinja2.
That probably is not what you want to do - it’s safer to specify an exact version of a package (aka version pinning) to protect from your dependencies breaking changes, which may happen in the future. For instance at the time of writing, aiohttp is available in 2.0.0rc1 so without version pinning Łapka might not run when installed after aiohttp 2.0.0 release. Here’s how to pin your dependencies:
But let’s say I’m developing some library. I shouldn’t be so specific about dependencies version - my library can run on a variety of releases. As an example let’s say I’m developing a aiohttp 1.2.0+-only plugin but not for 2.0.0 and future releases. How do I tell this in a requirements file? Something like this would do:
The same syntax for version pinning may be used in a command line, like:
$ pip install "aiohttp>=1.2,<2.0"
Mind extra quotes - in a shell environment some symbols (like
might have some syntax meaning - so it is essential to escape them.
Isn’t this section already covered? Partially it is - you can just run pip and point to it your requirements.txt. But what about dependencies of dependencies?
Let’s have a look on a another pip command,
freeze. Running it will list
all packages you currently have installed in your environment - including
dependencies of your dependencies. Now you can see, that after installing
aiohttp there are a few other libraries attached to the list, like chardet
or yarl. What if one of those packages future release will be incompatible or
buggy? It may break your application, especially if you are not using continues
integration! We need to add them to our requirements file as well and we need
to do it quickly! Now it looks like this:
aiohttp==1.3.3 aiohttp-jinja2==0.13.0 async-timeout==1.2.0 chardet==2.3.0 Jinja2==2.9.5 MarkupSafe==1.0 multidict==2.1.4 yarl==0.9.8
But… do we need to?
Adding all of the output from
pip freeze to our requirements file will cause
upgrading packages to be a pain. You probably want to install a security
patches to all packages and do it in a convenient way, don’t you?
Luckily, there is a better approach - using constraints file. This allows to keep your requirements.txt (very) simple thanks to storing all version information in a separate file.
So, requirements file now only contains:
and newly created constraints.txt now includes:
iohttp==1.3.3 aiohttp-jinja2==0.13.0 async-timeout==1.2.0 chardet==2.3.0 Jinja2==2.9.5 MarkupSafe==1.0 multidict==2.1.4 yarl==0.9.8
To install correct version of all packages just run:
$ pip install -r requirements.txt -c constraints.txt
Upgrading your packages is also super simple. Just run:
$ pip install --upgrade -r requirements.txt # or just upgrade a single package
and after verifying everything works just fine with updated dependencies (ie by running tests), generate constraints again:
$ pip freeze > constraints.txt
Don’t forget to include both requirements.txt and constraints.txt in your repository.
Please mind that if you install any additional packages it will be included (with its dependencies) into your constraints. How to prevent this from happening will be explained in the second part of this mini-series so stay tuned!
If you wish to learn more about pip itself, an excellent place would be its rich documentation. It includes an user guide, full commands, options and switches reference and many explanatory examples.