Conversation
VixDiskLib_QueryAllocatedBlocks (AIO type 13) returns which blocks within a disk are allocated (non-sparse), useful for skipping empty regions when reading a snapshot delta file. Reverse-engineered via an SSL/write-hook capture extended to a ctypes call to VixDiskLib_QueryAllocatedBlocks after Open, hitting and resolving two non-obvious wire-format bugs along the way: - Field-order swap: a first capture used startSector=0, which made two request fields (a reserved field and start_offset_bytes) both read as zero -- indistinguishable. Implementing from that single capture put start_offset_bytes at the wrong byte offset (8 instead of 24) and silently returned wrong (all-empty) results for every non-zero start. A second capture against a known-allocated region with a non-zero startSector broke the tie. - Bitmap padding: the reply's allocation bitmap is padded to a 4-byte boundary, not the raw ceil(chunk_count/8) an earlier draft assumed -- invisible for a chunk_count that's already a multiple of 4, but it under-read and desynced the connection for smaller chunk counts. Also investigated the "NFC_DELTA_DISK" backlog item (reading directly from a snapshot delta chain), expecting a distinct wire message. `strings` on libvixDiskLib.so found it's actually an alternate OPEN_FILE file-type value used by an internal VDDK client-side optimization for very sparse VMFS redo logs -- not a correctness requirement. Verified end-to-end that reading, writing, and query_allocated_blocks against an actual post-snapshot delta file all already work with the existing NFC_DISK-only implementation. Found and documented a real gotcha along the way: querying allocated blocks on the same still-open handle a write just went through can see stale (pre-write) data -- reproduced identically on native VDDK (two-process capture, since loading native VDDK in the same process as pyVmomi segfaults on this host's OpenSSL), so this is real server/VMFS behavior, not a client bug. Validated against a live standalone ESXi 8.0.3 host: full-disk query, a known-allocated sub-range, an aligned empty range, and a real snapshot delta file all match native VDDK's own output exactly. Adds unit tests for the bitmap decode/merge logic and input validation (no lab needed) plus integration tests for the live scenarios above. Full protocol details and both bugs are in docs/nfc_read.md; the NFC_DELTA_DISK investigation is in docs/reverse_engineering_procedure.md.
For better readability, we'll add an inline comment that describes how allocation bitmaps are parsed.
We'll use the same typing.Protocol approach as cloudbase#4, minimizing merge conflicts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VixDiskLib_QueryAllocatedBlocks(AIO type 13): returns which blocks within a disk are allocated (non-sparse). Reverse-engineered via an SSL/write-hook capture, hitting and resolving two non-obvious wire-format bugs:startSector=0, which made two request fields (a reserved field andstart_offset_bytes) both read as zero — indistinguishable. Implementing from that single capture putstart_offset_bytesat the wrong byte offset and silently returned wrong (all-empty) results for every non-zero start. A second capture against a known-allocated region with a non-zerostartSectorbroke the tie.ceil(chunk_count/8)an earlier draft assumed — invisible for achunk_countthat's already a multiple of 4, but it under-read and desynced the connection for smaller chunk counts.NFC_DELTA_DISKinvestigated: expected a distinct wire message for reading snapshot delta files.stringsonlibvixDiskLib.sofound it's actually an alternateOPEN_FILEfile-type value used by an internal VDDK optimization for very sparse VMFS redo logs — not a correctness requirement. Verified end-to-end that reading, writing, andquery_allocated_blocksagainst an actual post-snapshot delta file all already work with the existingNFC_DISK-only implementation.Full protocol details and both bugs:
docs/nfc_read.md. TheNFC_DELTA_DISKinvestigation:docs/reverse_engineering_procedure.md.Test plan
pytest tests/unittest_query_allocated_blocks,test_write_and_query_allocated_blocks_on_delta_disk) — the former initially had the same "write and query on one still-open handle" staleness bug documented above (caught while validating this split), fixed to use separate handlesNote on PR sequencing
Third of 4 focused PRs splitting up the original combined PR #2 (now closed). Independent of #3 (direct-ESXi fix) and #4 (
GetInfo+DDB_GET) — can merge in any order. (Validated live by temporarily stacking on the direct-ESXi fix locally, since our test lab is a bare ESXi host that needs that fix to connect at all.)🤖 Generated with Claude Code