Poetry

The following Poetry configuration is summarized from the excellent write up on Alex Mitelman’s website: https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/

Fast Track

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#fast-track

1
2
3
4
5
6
7


poetry new my-project; cd my-project; ls
pyenv local 3.9.2
poetry env use python
poetry add --dev pytest-cov pre-commit flake8 mypy isort
poetry add --dev --allow-prereleases black
poetry shell
code .

Add config to pyproject.toml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


[tool.isort]
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 79

[tool.black]
line-length = 79
target-version = ['py38']
include = '\.pyi?$'
exclude = '''

(
  /(
      \.eggs         # exclude a few common directories in the
    | \.git          # root of the project
    | \.hg
    | \.mypy_cache
    | \.tox
    | \.venv
    | _build
    | buck-out
    | build
    | dist
  )/
  | foo.py           # also separately exclude a file named foo.py in
                     # the root of the project
)
'''

Create setup.cfg:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


[flake8]
extend-ignore = E203

[mypy]
follow_imports = silent
strict_optional = True
warn_redundant_casts = True
warn_unused_ignores = True
disallow_any_generics = True
check_untyped_defs = True
no_implicit_reexport = True
disallow_untyped_defs = True
ignore_missing_imports = True

[mypy-tests.*]
ignore_errors = True

Create .pre-commit-config.yaml.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.4.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files
-   repo: https://gitlab.com/pycqa/flake8
    rev: 3.8.4
    hooks:
    -   id: flake8
-   repo: https://github.com/psf/black
    rev: 20.8b1
    hooks:
      - id: black
-   repo: https://github.com/pre-commit/mirrors-mypy
    rev: v0.812
    hooks:
        - id: mypy
          additional_dependencies: [pydantic]  # add if use pydantic
-   repo: https://github.com/PyCQA/isort
    rev: 5.7.0
    hooks:
    -   id: isort

Initialize the project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


echo '.coverage' > .gitignore
echo '.vscode/\n.idea/' >> .gitignore
curl -s https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore >> .gitignore
git init -b main
git add .
git commit -m 'Initial commit'
pre-commit install
pre-commit autoupdate
pre-commit run --all-files
pre-commit run --all-files

Components

Pyenv

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#how-to-manage-python-versions-with-pyenv

Install:

1
2
3


brew update
brew install pyenv
brew install zlib

add to ~/.bash_profile

1
2
3
4
5


export LDFLAGS="-L/usr/local/opt/zlib/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include"
export PATH="$HOME/.poetry/bin:$PATH"
export PATH="$HOME/.pyenv/shims:$PATH"
eval "$(pyenv init -)"

Use:

1
2
3


pyenv install --list | grep " 3\."
pyenv install -v 3.8.6
pyenv local 3.8.6

Poetry

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#dependency-management-for-python-with-poetry

1
2
3
4
5
6
7
8
9


curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
source $HOME/.poetry/env
poetry new my-project
cd my-project; ls
pyenv local 3.8.5
poetry env use python
poetry shell
python -V
code .

Run a command in the virtual environment:

1

poetry run python -V

Spawn a shell in the virtual environment:

1

poetry shell

Pytest

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#how-to-test-with-pytest

Poetry creates a test file automatically

1

poetry run pytest

or if in the environment shell, just

1

pytest

PytestCov

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#how-to-measure-tests-coverage-with-pytest-cov

1
2
3


poetry add --dev pytest-cov
echo '.coverage' > .gitignore
pytest --cov=my_project tests/

Git

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#how-to-run-checks-before-committing-changes-with-pre-commit

1
2
3


curl -s https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore >> .gitignore
echo '.vscode/\n.idea/' >> .gitignore
echo '.coverage' > .gitignore

GitPreCommit

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#why-run-checks-before-commit

1
2
3
4
5
6


poetry add pre-commit --dev
pre-commit sample-config > .pre-commit-config.yaml
pre-commit run --all-files
pre-commit install
git commit -m 'Initial commit'
pre-commit autoupdate

I ended up doing:

1

brew install pre-commit

Flake8

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#code-analysis-with-flake8-linter

1
2


poetry add flake8 --dev
flake8 .

How to add Flake8 to git hooks? To do this, open .pre-commit-config.yaml and add following:

1
2
3
4


-   repo: https://gitlab.com/pycqa/flake8
    rev: 3.9.2
    hooks:
    -   id: flake8

1
2
3


git add .pre-commit-config.yaml
git commit -m 'Add Flake8 to git hooks'
pre-commit run --all-files

Black

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#formatting-code-with-black

1

poetry add --dev black --allow-prereleases

To configure Black, let’s open pyproject.toml and add following section:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


[tool.black]
line-length = 79
target-version = ['py38']
include = '\.pyi?$'
exclude = '''

(
  /(
      \.eggs         # exclude a few common directories in the
    | \.git          # root of the project
    | \.hg
    | \.mypy_cache
    | \.tox
    | \.venv
    | _build
    | buck-out
    | build
    | dist
  )/
  | foo.py           # also separately exclude a file named foo.py in
                     # the root of the project
)
'''

We also have to suppress some error at Flake8 to make it work with Black. For this, we have to create setup.cfg file, which is a config file for Flake8, and put following in there:

1
2


[flake8]
extend-ignore = E203

add to .pre-commit-config.yaml:

1
2
3
4


-   repo: https://github.com/psf/black
    rev: 20.8b1
    hooks:
      - id: black

run:

1
2


black .
pre-commit run --all-files

Mypy

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#static-typing-with-mypy

1
2


poetry add --dev mypy
mypy .

Open setup.cfg file and add following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


[mypy]
follow_imports = silent
strict_optional = True
warn_redundant_casts = True
warn_unused_ignores = True
disallow_any_generics = True
check_untyped_defs = True
no_implicit_reexport = True
disallow_untyped_defs = True
ignore_missing_imports = True

To fix it, let’s add type annotation to our function:

1
2
3
4
5


from numbers import Real
from typing import Union

def multiply_two_numbers(a: Union[int, Real], b: Union[int, Real]) -> Union[int, Real]:
    return a * b

As we can see, function definition got too long. By pressing ⇧⌥F , VS Code formats code with Black to make it fit into 79 characters per line:

1
2
3
4
5
6
7


from numbers import Real
from typing import Union

def multiply_two_numbers(
    a: Union[int, Real], b: Union[int, Real]
) -> Union[int, Real]:
    return a * b

Obviously, we want to make sure that Mypy runs before committing the code. Add following to .pre-commit-config.yaml:

1
2
3
4
5


-   repo: https://github.com/pre-commit/mirrors-mypy
    rev: v0.782
    hooks:
        - id: mypy
          additional_dependencies: [pydantic]  # add if use pydantic

ISort

https://mitelman.engineering/blog/python-best-practice/automating-python-best-practices-for-a-new-project/#sorting-imports-with-isort

1
2


poetry add --dev isort
isort .

To make it works with Black correctly, we should add following to pyproject.toml:

1
2
3
4
5
6


[tool.isort]
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 79

As a final step, let’s add it to the hooks list in .pre-commit-config.yaml:

1
2
3
4


-   repo: https://github.com/PyCQA/isort
    rev: 5.4.2
    hooks:
    -   id: isort