Improving build performance of Sphinx documentation

When building multilingual documents using Sphinx, we made performance improvements as the build time became longer. In this article, I will summarize the situation before and after the improvement, the process of trial and error, and the results obtained.

---

background

I was using Sphinx with the following configuration:

• Multilingual configuration: ja, en

• Define the doc command in setup.py and build with python setup.py doc

• poetry + poe を使って poetry run poe doc で実行

• Initial build time: about 1 minute

---

Initial measurement (before improvement)

Measurements were made as follows.

time poetry run poe doc

output:

poetry run poe doc  57.11s user 2.96s system 97% cpu 1:01.38 total

At this stage, all languages ​​were fully built serially and with the -E -a option.

---

Trial of improvement measures

I tried the following improvements in order:

1. Removed -E -a (enables differential builds)

2. Parallel builds per language (via multiprocessing)

3. Difference support for static file copy (with judgment by filecmp)

4. Sphinx internal parallelization (-j auto option)

5. Verification of parallelism effect by number of cores and virtual CPU environment

Note

On Linux, you can check the CPU specs with the lscpu command.

---

Final configuration (best practices)

I ended up with a configuration like this:

• Remove -E -a and enable differential build

• I purposely do not use parallel build, and series is stable in a 2-core environment.

• Enable parallel processing inside Sphinx using -j auto

• Static file copying includes difference judgment

---

Measurement results after improvement

✨ 全ビルド完了。所要時間: 6.23 秒
poetry run poe doc  8.89s user 1.28s system 99% cpu 10.199 total

Build time has been reduced to less than 1/6th compared to before the improvement.

---

Discussion and future

• Sphinx can be greatly speeded up by using file cache (doctrees)

• The effect of parallelization is limited in environments with a small number of CPU cores.

• Internal parallelization with -j auto is effective

• If the number of languages ​​increases or the configuration of the virtual environment changes, consider introducing parallel processing again.

Article information

author:

Mr. Takagi

Post date:

2025-05-27

Measurement method for Sphinx build time (time command and Measure-Command)   Migrating from poethepoet to doit in a Python development environment and best practices