Skip to content

Sync the archive with Substack again and follow its new cursor paging - #39

Merged
Varnasr merged 2 commits into
mainfrom
ccr-20dec548-7a2zpz
Nov 6, 2026
Merged

Varnasr merged 2 commits into
mainfrom
ccr-20dec548-7a2zpz

Conversation

@Varnasr

@Varnasr Varnasr commented Nov 6, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

Substack's archive endpoint began answering HTTP 400 to offset and to a limit above 25, and returns {posts, nextCursor} instead of a list, so scripts/sync_substack.py could no longer list posts. The archive was six weeks behind (it stopped at 7 September).

  • list_posts pages by cursor with limit=25, and still accepts a bare list in case Substack reverts.
  • The archive gains 36 posts (up to 17 October 2026), and three existing posts change title or subtitle to match Substack. Companion pages are rebuilt and match the archive.
  • Tests cover the cursor paging and fail if offset is ever sent again. CLAUDE.md records what is known about the failure.

Type of change

  • Bug fix (broken link, layout, JS error)
  • New content (course, case study, translation)
  • New feature or tool
  • Accessibility improvement
  • Performance improvement
  • Documentation update

Checklist

  • Tested on desktop browser (no page change; pytest: 34 passed; build_companions.py --check: PASS)
  • Tested on mobile browser
  • No console errors
  • Links are working

Note: the monthly sync workflow opens its own PR, so this should merge first.


Generated by Claude Code

Substack's archive endpoint began answering HTTP 400 to offset and to a limit
above 25, and returns {posts, nextCursor} instead of a list, so the sync had
failed since early September. list_posts now pages by cursor (limit=25). The
archive gains 36 posts and three titles and subtitles change as Substack has
them; companions are rebuilt and match the archive. Tests cover the cursor
paging and fail if offset is ever sent again.

pytest: 34 passed. build_companions.py --check: PASS.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011RjaPcC924ePsEE8X6uH72
@Varnasr
Varnasr merged commit 31218ca into main Nov 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants