The backpressure mechanism is currently triggered by the number of loose batches in the spine. This ignores the potentially large number of batches being merged, which are still part of the spine until the merges finish. Arguably, total batch count is a better trigger.
The backpressure mechanism is currently triggered by the number of loose batches in the spine. This ignores the potentially large number of batches being merged, which are still part of the spine until the merges finish. Arguably, total batch count is a better trigger.