In the first part of this mini-series I would like to explain how an user can:

  • install a Python package,
  • pin version of dependency,
  • save dependencies for later.

You can also browse other posts from this mini-series:

Installing packages

Python’s default package manager is called pip which replaced a previous one, easy_install. Basic usage of pip is simple: pip install this, pip uninstall that is most of what you will do on a daily routine. But don’t be disappointed - pip’s commands are flexible thanks to a number of switches and options available.

One of the most common switches is, from my experience, --requirement (or -r for shortcut). It allows an user to install packages referenced in a requirements file. Pip allows -r option to be used multiple times so there might (and probably - should) be distinct files for running a project and developing (or testing) it. Pip will install wheel releases, if available, by default. Wheels are Python’s prebuild packages, thanks to them you don’t need to compile a plugin written in C on your dev machine or a server. To learn more about wheels go to packaging.python.org.

Version pinning and requirements file

In the previous section I’ve mentioned requirements file. How to prepare one and what is its format?

It is just a regular text file, usually with a .txt extension, which list packages names you want to install. For instance such a requirements file:

aiohttp
aiohttp_jinja2

will tell pip to install the latest versions of aiohttp and aiohttp_jinja2.

That probably is not what you want to do - it’s safer to specify an exact version of a package (aka version pinning) to protect from your dependencies breaking changes, which may happen in the future. For instance at the time of writing, aiohttp is available in 2.0.0rc1 so without version pinning Łapka might not run when installed after aiohttp 2.0.0 release. Here’s how to pin your dependencies:

aiohttp==1.3.3
aiohttp_jinja2==0.13.0

But let’s say I’m developing some library. I shouldn’t be so specific about dependencies version - my library can run on a variety of releases. As an example let’s say I’m developing a aiohttp 1.2.0+-only plugin but not for 2.0.0 and future releases. How do I tell this in a requirements file? Something like this would do:

aiohttp>=1.2,<=2.0

The same syntax for version pinning may be used in a command line, like:

$ pip install "aiohttp>=1.2,<2.0"

Mind extra quotes - in a shell environment some symbols (like =, < or >) might have some syntax meaning - so it is essential to escape them.

Saving dependencies

Isn’t this section already covered? Partially it is - you can just run pip and point to it your requirements.txt. But what about dependencies of dependencies?

Let’s have a look on a another pip command, freeze. Running it will list all packages you currently have installed in your environment - including dependencies of your dependencies. Now you can see, that after installing aiohttp there are a few other libraries attached to the list, like chardet or yarl. What if one of those packages future release will be incompatible or buggy? It may break your application, especially if you are not using continues integration! We need to add them to our requirements file as well and we need to do it quickly! Now it looks like this:

aiohttp==1.3.3
aiohttp-jinja2==0.13.0
async-timeout==1.2.0
chardet==2.3.0
Jinja2==2.9.5
MarkupSafe==1.0
multidict==2.1.4
yarl==0.9.8

But… do we need to?

Adding all of the output from pip freeze to our requirements file will cause upgrading packages to be a pain. You probably want to install a security patches to all packages and do it in a convenient way, don’t you?

Luckily, there is a better approach - using constraints file. This allows to keep your requirements.txt (very) simple thanks to storing all version information in a separate file.

So, requirements file now only contains:

aiohttp
aiohttp_jinja2

and newly created constraints.txt now includes:

iohttp==1.3.3
aiohttp-jinja2==0.13.0
async-timeout==1.2.0
chardet==2.3.0
Jinja2==2.9.5
MarkupSafe==1.0
multidict==2.1.4
yarl==0.9.8

To install correct version of all packages just run:

$ pip install -r requirements.txt -c constraints.txt

Upgrading your packages is also super simple. Just run:

$ pip install --upgrade -r requirements.txt  # or just upgrade a single package

and after verifying everything works just fine with updated dependencies (ie by running tests), generate constraints again:

$ pip freeze > constraints.txt

Don’t forget to include both requirements.txt and constraints.txt in your repository.

Please mind that if you install any additional packages it will be included (with its dependencies) into your constraints. How to prevent this from happening will be explained in the second part of this mini-series so stay tuned!

If you wish to learn more about pip itself, an excellent place would be its rich documentation. It includes an user guide, full commands, options and switches reference and many explanatory examples.