Repository navigation
Version registry skips a release when the listing shifts under the stored tail #159
Description
Description
The version registry loads older releases on demand in StoredVersionRegistry::extend() with $source->pages($record->count()): the number of stored releases is taken as the position of the next one in the provider's listing. GitHubReleaseSource and GitLabReleaseSource then turn that offset into a page number assuming 100 releases per page (intdiv($offset, 100) + 1, $offset % 100).
Both assumptions break, with different costs.
1. A release is lost when one is deleted in the stored tail (correctness)
RepositoryRecord::withHead() drops releases deleted upstream only within the span the last check() fetched (usually the first page). A deletion further down leaves a ghost in the record, so count() is higher than the real position, and extend() starts past a release that was never stored.
Reproduced on github.com semantics (100 per page):
- The repository has 300 releases. A run stores 200 of them and stops (the record is incomplete).
- Upstream, release feat(version): match constraints with DLoad's own version ordering #150 is deleted — outside the first page covered by
check(). - The next run with
--refreshrequestspage=1, thenextend()asks for offset 200 →page=3. The release that moved from position 201 to 200 (v1.0.100in the repro) is skipped. - The record is marked complete, so later runs never see that release; the deleted
v1.0.149stays in the list.
The same happens when draft placeholders disappear (token changed, draft deleted): they are stored precisely to keep positions. forget() after a 404 lowers count() too, but that only makes the next page overlap, which is harmless.
2. A server that ignores per_page refetches pages (cost)
A mirror, a fake GitHub or another self-hosted server (more likely with the server attribute of #148) may return fewer than 100 releases per page. The offset→page translation then lands on an earlier page: on the first run page 1 is requested twice, and on every later run all stored pages are requested again (20 requests instead of 10 with 20 stored pages of 2), since fully known pages yield nothing and the consumer does not stop. GitLab is affected the same way as GitHub.
Root cause
The registry derives a listing position from its local count(), which holds only while the page size is 100 and nothing above the tail has been deleted since it was stored.
Expected
extend() continues exactly after the last stored release, whatever was deleted above it and whatever page size the server uses; a shift in the listing is detected rather than silently skipped. Ideally no extra requests in the common case (nothing shifted, 100 per page).
Additional context
Found while reviewing the note about the release page offset. Code: src/Module/Registry/Internal/StoredVersionRegistry.php (check(), extend()), src/Module/Registry/Record/RepositoryRecord.php (withHead(), withTail()), src/Module/Repository/Internal/{GitHub,GitLab}/*ReleaseSource.php.
Activity