Skip to content

receive: a 202 while the server prepares reports complete: true #80

Description

@andinux

Summary

While the server is still preparing a download, cloudsync_network_receive_changes() reports the sync as complete:

{"receive":{"rows":0,"tables":[],"chunks":0,"bytes":0,"complete":true}}

complete is documented as "true iff the receive stream is fully drained (nothing pending)" (src/network/network.c:2297), so an application that stops on it stops too early and waits for its next sync tick.

This predates the window cap work and is independent of it.

Why it happens

network_drain_changes initialises bool complete = true (network.c:2465). A 202 leaves it untouched:

// nothing delivered (202 / up to date): preserve the polling-for-changes semantics.
// complete is left as-is (true if no page was ever delivered; false if the last
// delivered page was non-final), so a 202 after partial pages reports incomplete.
if (ntries + 1 >= max_retries) break;

The comment is accurate about the intent, but the first call of a drain has never delivered a page, so complete is still true.

The underlying cause is that "preparing" and "up to date" are indistinguishable to the client: both arrive as a response carrying neither url nor payload (network.c:2225-2235), and both produce rows: 0.

The two entry points differ in how exposed they are:

  • cloudsync_network_receive_changes() passes wait_ms=0, max_retries=1 (:2612), so it breaks on the first 202 without sleeping and reports complete: true immediately.
  • cloudsync_network_sync() passes the caller's wait_ms/max_retries (:2547), so it polls and usually gets the data within the same call — it only reports complete: true if preparation outlasts the retry budget.

Where it bites

A first sync of a large database, which is exactly when preparation is slowest. A fresh site's first /check makes the server spool the whole tenant history and answer 202 until ready; on a large tenant that is tens of seconds. An application calling cloudsync_network_receive_changes() in that window is told it is up to date while none of its data has arrived.

Options

  1. Distinguish the two states on the wire. The server knows whether a prepare job is queued or running for the requesting checkpoint (it already looks this up to decide whether to enqueue one). An additive response field would let the client report complete: false while preparing. Costs a lookup on the most frequent response the server gives, so worth weighing.
  2. Client-side only: report complete: false when a drain ends on a 202 having delivered nothing and the local checkpoint is behind a known server version. Needs a server-side version to compare against, which the check response does not currently carry.
  3. Document it and leave it, making clear that complete: true with rows: 0 means "nothing was delivered", not "nothing remains", and that callers should use cloudsync_network_sync() with a retry budget when they need certainty.

Option 3 is honest but weak for the case above, where the caller has no way to tell how long to keep asking.

Not urgent for 1.2.0

Nothing in the window-cap work depends on this, and the behaviour is unchanged from previous releases. Filed so it is tracked rather than rediscovered: it was found while designing the cap, where a capped window would have made this state routine rather than rare.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions