Skip to content

Conversation

@patrickwwbutler
Copy link
Contributor

Updates the segment.bytes config value to 128MiB to avoid having to process large amounts of progress data when creating new sinks.

Motivation

  • This PR adds a known-desirable feature.

https://github.com/MaterializeInc/database-issues/issues/9894

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

@patrickwwbutler patrickwwbutler force-pushed the patrick/kafka_progress branch 2 times, most recently from dcb62be to bd21cf6 Compare December 11, 2025 19:51
@patrickwwbutler patrickwwbutler marked this pull request as ready for review December 11, 2025 20:19
@patrickwwbutler patrickwwbutler requested a review from a team as a code owner December 11, 2025 20:19
Copy link
Contributor

@martykulma martykulma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks @patrickwwbutler !

There's also a section for the progress topic in the docs that could use a mention of the segment size: https://materialize.com/docs/serve-results/sink/kafka/#automatic-topic-creation

@patrickwwbutler patrickwwbutler requested a review from a team as a code owner December 12, 2025 15:05
Copy link
Member

@DAlperin DAlperin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

Copy link
Contributor

@martykulma martykulma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐟

@patrickwwbutler patrickwwbutler merged commit 8a78d2b into MaterializeInc:main Dec 13, 2025
131 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants