Getting started with Python Poetry

posted on 2021-09-13

Poetry will enable you to easily manage dependencies, create packages and manage your development virtual environment.

In this blog post I'll give you a quick introduction to get you started using Poetry to manage your Python project.

Why use Poetry

Whenever you write a program, there will come a time you start to require external modules to solve common problems. For example, use the requests library to make HTTPS requests.

You can easily install your requests manually, however if you want to later share your code with other people or run your code on other systems, you will have to install requests there manually again as well.

What if you have one program that requires requests version 1 and another which requires requests version 2? You can't have both version 1 and version 2 on the same PYTHONPATH, so you will have to start using virtual environments.

Another issue is making sure that all the libraries you want to use, actually work together. If you install requests 2.6 and then install urllib3 1.8 you will actually end up with a broken environment: requests wants urllib3 above version 1.21.1. Solving this puzzle is best automated, and every modern language has a companion build tool to solve this dependency puzzle for you (cargo, yarn, stack, etc.).

For Python there are pipenv and poetry. Let's dive into how to use poetry.

Installing Poetry

First install poetry. If you have a package manager, use that. If not, you can install poetry using pip install poetry. For more information see the official documentation.

After installation, I advise to use a local in project environment configuration. Configure this by running

poetry config virtualenvs.in-project true

Poetry by example

Poetry projects are managed using two configuration files: the pyproject.toml and poetry.lock. The pyproject.toml is a file you can edit manually, the poetry.lock describes the exact versions found after solving the dependency puzzle and is fully managed by Poetry. Do not edit/touch this file, but do add it to your git repository.

In the next steps we create a tiny program using Poetry from scratch. The program will simply check if there is a holiday somewhere and we will call it where is the party (witp).

Start a new project by running

poetry new witp

Open the witp/__init__.py and put the whole implementation in there:

import datetime
from importlib.metadata import version
from typing import Optional, Tuple

import holidays

__version__ = version(__name__)


def main():
    country, holiday = where_is_the_party(datetime.date.today()) or (None, None)
    if country is not None:
        print(f"Go pack your bags, we need to go to {country} to celibrate {holiday}")
    else:
        print("Can't find the party, sorry")


def where_is_the_party(today: datetime.date) -> Optional[Tuple[str, str]]:
    for country in holidays.list_supported_countries():
        ch = holidays.CountryHoliday(country=country)
        if ch.get(today):
            return country, ch.get(today)
    return None

To be able to run this program, we make sure there is an script to run it as a commandline utility. To do this, we add some configuration to the pyproject.toml file:

[tool.poetry.scripts]
witp = 'witp:main'

By adding those lines at the end, poetry install will create an executable called witp which runs the main program. Because this is inside the virtual environment, we have poetry run the command inside the environment:

poetry run witp

Oh oh.. we forgot about the holiday library and get

ModuleNotFoundError: No module named 'holidays'

Ok, so let's ask poetry to add that dependency to the pyproject.toml:

poetry add holidays

This will update the environment. So now if we run

poetry run witp

we get:

Can't find the party, sorry

Hmmm... will it every find a party? Let's add a test to make sure we are doing the right thing. Open the tests/test_witp.py and replace the contents with:

import datetime

from witp import where_is_the_party


def test_holiday():
    assert where_is_the_party(datetime.date(2020, 12, 25)) == (
        "ABW",
        "Pasco di Nacemento [Christmas]",
    )

Now let's run those tests and see if it works:

poetry run pytest

All green? Great! But a bit boring, and this is about having a fun project. So let's add a smiley to that result.

poetry add --dev pytest-emoji

Using the --dev flag, we make sure that the dependency is not installed if somebody wants to use our package, only when they are doing development on the project.

poetry run pytest --emoji

Great, a smiley to celebrate the result!

Ok, so we have this great program, we need to share it with the world. Let's create a python package we can e-mail to a friend:

poetry build

Now we have a package in the dist folder called witp-0.1.0-py3-none-any.whl. We can run pip install witp-0.1.0-py3-none-any.whl wherever we need it.

Summary

We create a project and package using the following commands:

poetry config to configure poetry for the first time.
poetry new to get started with a new directory.
poetry run to execute a command in the virtual environment. We used pytest but we could just as well have executed python.
poetry add to add a dependency to the project.
poetry add --dev to add a dependency for development only.
poetry build to create a package.

There is way more you can do and learn when it comes to pyproject.toml configuration and how to distribute this package on pypi for example, but this should get you up and running for your first project using poetry.

Happy hacking!