Migrating from poethepoet to doit in a Python development environment and best practices

In my Python project using Poetry, I used poethepoet to manage tasks, but I decided to migrate to doit for the following reasons.

• Allows for more flexible dependency management and customization

• Easily supports multiple task dependencies and parallel execution

• I wanted to organize Sphinx multilingual builds, static file copies, etc.

• I wanted to integrate the custom commands written in setup.py into dodo.py

Background and current situation

In the traditional configuration, tasks were managed in the following way:

• poetry run poe doc → python setup.py doc

• poetry run poe gettext → python setup.py gettext

• Define doc / gettext command class in setup.py

• Wrapped in [tool.poe.tasks] in pyproject.toml

Although this configuration worked reasonably well, it also had the following challenges:

• Depends on setup.py (not recommended)

• There are limits to flexible task definition (file dependence/re-execution determination)

• Cannot be used in conjunction with the introduction of pre-commit

Steps to transition to doit

1. First add doit as a development dependency:

   poetry add -D doit
   

2. Create dodo.py in the project root and define the task as below:

   from doit import create_after
   import subprocess, shutil, os, time, filecmp
   
   LANGUAGES = ['ja', 'en']
   def task_doc():
       def run():
           for lang in LANGUAGES:
               subprocess.run([
                   "sphinx-build", "-b", "html", "-D", f"language={lang}",
                   "docs", f"docs/_build/html/{lang}", "-d", f"docs/_build/doctrees_{lang}"
               ], check=True)
           shutil.copy("static/index.html", "docs/_build/html/index.html")
       return {'actions': [run]}
   

3. Operation confirmation:

   poetry run doit list
   poetry run doit doc
   

4. If there is no problem, delete poe:

   poetry remove poe
   

   Also, delete the [tool.poe.tasks] section from pyproject.toml.

5. Since .doit.db.db is not managed by Git, add the following to .gitignore:

   .doit.db.db
   

Use with pre-commit

Changed code format and static analysis to be done in pre-commit.

repos:
  - repo: https://github.com/psf/black
    rev: 25.1.0
    hooks:
      - id: black
  - repo: https://github.com/pre-commit/mirrors-isort
    rev: v5.10.1
    hooks:
      - id: isort
  - repo: https://github.com/PyCQA/flake8
    rev: 7.2.0
    hooks:
      - id: flake8

Summary of advantages

• doit is based on Python, has high extensibility, and has excellent re-execution judgment.

• setup.py was abolished and the structure was organized around pyproject.toml

• Uniformly manage pre-commit checks with pre-commit

• Task execution result cache is also optimized using .doit.db.db

Article information

author:

Mr. Takagi

Post date:

2025-05-28

Improving build performance of Sphinx documentation   Comment out selection range in Neovim: Introduce comment.nvim with vim-plug