Skip to content

feat(server): return base_offset and partition_id from send_messages#3101

Open
sujitn wants to merge 2 commits intoapache:masterfrom
sujitn:feat/send-messages-return-offset
Open

feat(server): return base_offset and partition_id from send_messages#3101
sujitn wants to merge 2 commits intoapache:masterfrom
sujitn:feat/send-messages-return-offset

Conversation

@sujitn
Copy link
Copy Markdown

@sujitn sujitn commented Apr 12, 2026

The offset assigned during append was computed but discarded. Propagate it through ShardResponse::SendMessages and include it in the binary protocol response.

Wire format change:
Before: [status:4 LE][length:4 LE (0)]
After: [status:4 LE][length:4 LE (12)][base_offset:8 LE][partition_id:4 LE]

existing SDK clients (excluding node) read the length field and skip unknown body bytes. The send_messages wrapper discards the response body and returns Ok(()).

Fixed node SDK to ignore length

Ref: #3044

Local Execution

  • Passed
  • Pre-commit hooks ran

AI Usage

If AI tools were used, please answer:

  1. Which tools?
    Claude

  2. Scope of usage?
    SDK validation

  3. How did you verify the generated code works correctly?

  • 64/64 server unit tests pass (cargo test -p server --lib)
  • 36/36 offset retrieval integration tests pass, these send messages via the SDK
  • 87/87 concurrent produce/consume integration tests pass
  • 6/6 authentication integration tests pass
  • cargo clippy -p server --all-targets -- -D warnings - 0 warnings
  1. Can you explain every line of the code if asked?
    Yes

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.73%. Comparing base (aa34566) to head (c130f35).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3101      +/-   ##
============================================
- Coverage     72.75%   72.73%   -0.02%     
  Complexity      943      943              
============================================
  Files          1117     1117              
  Lines         96343    96335       -8     
  Branches      73544    73552       +8     
============================================
- Hits          70091    70067      -24     
+ Misses        23704    23703       -1     
- Partials       2548     2565      +17     
Components Coverage Δ
Rust Core 73.50% <ø> (-0.01%) ⬇️
Java SDK 62.30% <ø> (ø)
C# SDK 69.10% <ø> (-0.31%) ⬇️
Python SDK 81.43% <ø> (ø)
Node SDK 91.53% <100.00%> (ø)
Go SDK 38.97% <ø> (ø)
Files with missing lines Coverage Δ
.../binary/handlers/messages/send_messages_handler.rs 100.00% <ø> (ø)
core/server/src/http/http_shard_wrapper.rs 74.12% <ø> (-0.18%) ⬇️
core/server/src/shard/handlers.rs 77.83% <ø> (-0.12%) ⬇️
core/server/src/shard/system/messages.rs 87.84% <ø> (+0.08%) ⬆️
core/server/src/shard/transmission/frame.rs 100.00% <ø> (ø)
foreign/node/src/client/client.utils.ts 89.89% <100.00%> (ø)

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sujitn sujitn force-pushed the feat/send-messages-return-offset branch 2 times, most recently from b3ad595 to ee34994 Compare April 12, 2026 13:26
@sujitn sujitn force-pushed the feat/send-messages-return-offset branch 2 times, most recently from 265d6be to d2b2e3d Compare April 13, 2026 05:51
The offset assigned during append was computed but discarded.
Propagate it through ShardResponse::SendMessages and include
it in the binary protocol response.

Wire format change:
  Before: [status:4 LE][length:4 LE (0)]
  After:  [status:4 LE][length:4 LE (12)][base_offset:8 LE][partition_id:4 LE]

Backwards compatible — existing SDK clients read the length field
and skip unknown body bytes. The send_messages wrapper discards
the response body and returns Ok(()).

Ref: apache#3044
@sujitn sujitn force-pushed the feat/send-messages-return-offset branch from 703510e to a685301 Compare April 13, 2026 18:08
@sujitn sujitn requested a review from hubcio April 13, 2026 21:17
@hubcio
Copy link
Copy Markdown
Contributor

hubcio commented Apr 13, 2026

I'm gonna review this properly once we finalize current release 0.8.0.

@lukaszzborek
Copy link
Copy Markdown
Contributor

I think we should at least create issues for every sdk to implement that change in api (just after merge)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants