Skip to content

Fix Base64 position wrapping after 3 GiB - #1516

Open
carrerasdarren-cell wants to merge 1 commit into
google:masterfrom
carrerasdarren-cell:agent/fix-base64-position-wrap
Open

Fix Base64 position wrapping after 3 GiB#1516
carrerasdarren-cell wants to merge 1 commit into
google:masterfrom
carrerasdarren-cell:agent/fix-base64-position-wrap

Conversation

@carrerasdarren-cell

Copy link
Copy Markdown

Base64 regions are recorded in Brotli's wrapped ring-buffer coordinate space,
but WriteMetaBlockInternal() currently passes the unwrapped 64-bit
last_flush_pos to the metablock and storage helpers. Once a streaming input
reaches the 3 GiB position-wrap boundary, a newly detected Base64 region near
the wrapped 1 GiB position appears to be behind the unwrapped current position.
It is consequently mapped to literal offset zero, and the Base64 histogram is
applied to unrelated bytes at the start of the metablock.

The encoder reports success, but the official decoder rejects the resulting
stream with BROTLI_DECODER_ERROR_FORMAT_PADDING_2.

This change restores the wrapped last_flush_pos used before Base64 mode was
introduced and passes that consistent coordinate to the ring-buffer,
metablock, and storage helpers.

The defect is limited to the opt-in BROTLI_PARAM_BASE64_MODE and streams that
cross the 3 GiB internal position boundary. It is distinct from #1509 and the
histogram-count fix in 0d1f629; current master after that fix still reproduces
this position mismatch.

Validation on current master 8e10eeb3378f6c459dbaf033ca6727e9816afccb:

  • A constant-memory public streaming-API reproducer round-trips the crafted
    stream one 1 MiB block below the boundary.
  • The same reproducer fails deterministically at exactly 3 GiB on untouched
    source and round-trips after this change.
  • The zero-offset control and both boundary controls pass under ASan/UBSan.
  • The complete CMake/CTest matrix passes, 73/73.
  • git diff --check passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant