A slow test suite does not merely waste time. It changes what engineers do: batch several changes into one pull request to amortise the wait, skip running tests locally, context-switch while waiting and lose the thread, and — most damaging — avoid refactoring because the verification cost is too high. The suite stops being a safety net and becomes a tax on improving the code.
The target worth aiming at is under ten minutes from push to merge signal, and under a minute for the tests relevant to the file you are editing. Both are achievable on large codebases, and neither requires deleting tests.
Run less, not worse
The largest win is usually test selection: determine which tests could possibly be affected by the changed files and run only those on the pull request, with the full suite on the main branch. Dependency-graph-based selection is exact for well-structured codebases; a coverage-map heuristic is a good approximation for the rest.
jobs:
fast:
steps:
# Lint and types first, in parallel, and fail the pipeline in 40 seconds
# rather than 14 minutes if someone left a syntax error.
- run: ruff check . & mypy src/ & wait
# Only the tests reachable from what changed.
- run: |
CHANGED=$(git diff --name-only origin/main...HEAD)
pytest $(python tools/affected_tests.py $CHANGED) -n auto --dist loadfile
full:
if: github.ref == 'refs/heads/main'
steps:
# The complete suite runs post-merge, split across machines by timing data
# so every shard finishes at roughly the same moment.
- run: pytest --splits 8 --group ${{ matrix.group }} --durations-path .test_durationsSplitting by recorded duration rather than by file count is a detail with outsized effect. Naive sharding leaves one machine running the four slowest integration tests while seven sit idle, and your pipeline takes as long as the unluckiest shard.
The rest of the budget
- Cache dependencies and build artifacts on a content hash. Reinstalling the same packages on every run is the most common wasted minute in CI.
- Start containers once per job, not per test. Database setup repeated across test classes is usually the hidden majority of the runtime.
- Fix flaky tests as incidents, not annoyances. A suite that fails randomly trains everyone to re-run rather than investigate, which disables the whole mechanism.
- Push slow end-to-end tests to a post-merge or nightly stage, and keep the pre-merge gate to what genuinely blocks a bad change.
- Measure and publish the p50 and p95 pipeline duration. What is not measured drifts upward one test at a time.
Engineers do not decide to stop refactoring. They just stop, because verifying a refactor costs forty minutes and nobody has forty minutes.
One reframing helps when arguing for the investment: the pipeline runs on every change from every engineer, several times a day. Cutting it from twenty minutes to six on a team of fifteen returns more engineering hours per month than most feature work consumes — and unlike feature work, the benefit compounds, because faster verification makes people willing to make the small improvements they currently skip.