Skip to content

Reduce public forum catalog latency with SQL pagination and batched access checks - #17

Merged
jmgasper merged 1 commit into
developfrom
codex/forum-catalog-latency-develop
Oct 7, 2026
Merged

jmgasper merged 1 commit into
developfrom
codex/forum-catalog-latency-develop

Conversation

@jmgasper

@jmgasper jmgasper commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Public forum requests were loading and enriching the entire catalog before pagination and repeatedly querying category permissions. After the legacy import, this saturated the dev service and contributed to gateway timeouts. This change selects authorized topic IDs in SQL before enriching only the requested page, batches category ancestry/access checks, and computes category statistics in PostgreSQL with at most five participant previews per category.

Visibility, direct topic roles, search, watches, sorting, and unread state remain permission-aware. Counts remain available beyond the last page, including oversized page numbers. Real PostgreSQL/HTTP regression coverage checks guest/member/imported-role/admin access, restricted and deleted descendants, counts, pagination, search, watches, and personalized unread state.

This branch is based directly on develop and contains the catalog implementation, tests, and documentation. Earlier Jive migration tools and dependency changes are excluded. No database migration is required.

Validation:

  • pnpm lint and pnpm build passed using the project Node version.
  • TZ=UTC PUBLIC_FORUMS_TEST_DATABASE_URL=postgresql://postgres@127.0.0.1:55439/catalog_latency_http_test pnpm test --runInBand: 21 suites / 170 tests passed; one optional suite / three tests skipped. UTC is documented because the existing import provenance test compares PostgreSQL timestamps without time zones.
  • Catalog application files and integration tests match the implementation already deployed as forums-api-v6:27. The live guest/member audit verified all 844/854 visible legacy categories, topic/post counts, pagination, and private-thread denial.
  • Sampled dev latency improved from 14.6s to 1.4s for a 10-topic page, and from 10.9s to 4.7s for categories. These are individual live checks, not a load benchmark; the full category tree remains approximately 700 KB.

The deployment notes also record the separately applied dev infrastructure changes: two healthy ECS tasks and an ALB default JSON 404 for unmatched requests, with existing routes preserved. Monitoring and alarm configuration are unchanged.

@jmgasper
jmgasper merged commit 77a37b6 into develop Oct 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant