Skip to content

Fix N+1 query in evolution chain serializer - #1688

Merged
FallenDeity merged 3 commits into
PokeAPI:masterfrom
santichausis:fix/evolution-chain-nplus1-query
Sep 30, 2026
Merged

FallenDeity merged 3 commits into
PokeAPI:masterfrom
santichausis:fix/evolution-chain-nplus1-query

Conversation

@santichausis

@santichausis santichausis commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

EvolutionChainDetailSerializer.build_chain_link_entry recurses over the evolution tree and, for every non-root species, ran its own PokemonEvolution.objects.filter(evolved_species=...) query with ~13 select_related joins. A linear 3-stage chain fires 2 of these queries; Eevee's 9-species chain fires 8.

Fix

Fetch all PokemonEvolution rows for the whole chain in a single query (filter(evolved_species__evolution_chain=obj)), group them in memory by evolved_species_id, and look them up from that dict during the recursion instead of querying per node. The query only runs when some species in the chain evolves from another, so single-species chains still issue no evolution query at all. The pk ordering (already applied by PokeApiManager) is spelled out explicitly at the call site.

Performance impact

Measured against the full dataset (DB built from the CSVs, all 540 evolution chains serialized with the old and new code):

  • Output: byte-identical JSON for all 540 chains; OpenAPI schema unchanged.
  • Queries: 1106 → 953 total (−14%). 134 chains need fewer queries (e.g. Eevee's 9-species chain: 9 → 2), none need more.
  • The 65 queries for Milcery → Alcremie are not addressed here. They come from per-row lookups in PokemonEvolutionSerializer (condition_expression / allowed_natures), handled separately in Avoid per-row queries for evolution condition variables and natures #1690.
  • In production, cachalot + Redis serves repeated queries from cache, so this mainly helps cold-cache requests.

Test plan

  • test_evolution_chain_api_query_count_does_not_scale_with_chain_size: a 1-branch and a 6-branch chain issue the same number of queries (fails with 8 != 3 without the fix).
  • test_evolution_chain_api_single_species_chain_skips_evolution_query: a single-species chain issues no PokemonEvolution query.
  • test_evolution_chain_api_evolution_details_order_with_many_rows_for_same_species: evolution_details keeps creation order for a species with several evolution rows.
  • manage.py test pokemon_v2 passes (66 tests); ruff check, ruff format --check and ty pass.

build_chain_link_entry ran a fresh PokemonEvolution query (with ~13
select_related joins) for every non-root species while recursing
through the evolution tree, so branching chains like Eevee's fired
one query per branch instead of one query for the whole chain.

Fetch all PokemonEvolution rows for the chain once, group them by
evolved_species_id, and look them up from that dict during the
recursion instead of querying per node.
Comment thread pokemon_v2/serializers.py Outdated
Chains with no evolutions (199 of 540 in the current data) don't need
the batched PokemonEvolution query at all; the old per-node code ran
zero queries for them, so the unconditional batch query made those
requests one query heavier. Only run it when some species in the
chain evolves from another.

Also make the pk ordering explicit on the batched query. PokeApiManager
already orders by pk, so this doesn't change behavior; it just keeps
the ordering visible at the call site. Add tests for both: a
single-species chain issues no PokemonEvolution query, and
evolution_details keeps creation order for a species with several
evolution rows (e.g. Milcery -> Alcremie).
@FallenDeity
FallenDeity merged commit bc92d3b into PokeAPI:master Sep 30, 2026
22 checks passed
@pokeapi-machine-user

Copy link
Copy Markdown

A PokeAPI/api-data refresh has started. In ~45 minutes the staging branch of PokeAPI/api-data will be pushed with the new generated data.

The staging branch will be deployed in our staging environment and the entire API will be ready to review.

A Pull Request (master<-staging) will be also created at PokeAPI/api-data and assigned to the PokeAPI Core team to be reviewed. If approved and merged new data will soon be available worldwide at pokeapi.co.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants