Skip to content

feat(store): add schema-based filesystem event loading - #520

Draft
johanpel wants to merge 5 commits into
rapidsai:mainfrom
johanpel:schema-based-event-import-iterator-store
Draft

feat(store): add schema-based filesystem event loading#520
johanpel wants to merge 5 commits into
rapidsai:mainfrom
johanpel:schema-based-event-import-iterator-store

Conversation

@johanpel

@johanpel johanpel commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Description

Add backend-neutral event-store traits with an initial filesystem implementation, plus quent-store-build for generating compatible model and entity markers from a schema.

Entity-specific loading is available by default. Enabling umbrella_event adds model-wide loading through the generated umbrella event type. Both APIs return unordered event iterators.

The standalone example demonstrates generating an analysis model and loading instrumentation output. CI covers the example independently.

Related Issues

Stacked on #519 so that needs to merge first

Written by Codex.

@johanpel
johanpel force-pushed the schema-based-event-import-iterator-store branch 3 times, most recently from f717f85 to 23584bf Compare August 5, 2026 06:35
@johanpel johanpel changed the title feat(io): add typed filesystem event stores feat(store): add schema-based filesystem event loading Aug 5, 2026
@johanpel
johanpel force-pushed the schema-based-event-import-iterator-store branch from 23584bf to 93ee16f Compare August 5, 2026 11:37
rapids-bot Bot pushed a commit that referenced this pull request Aug 6, 2026
# Description

Make importers yield `Result` items so read, framing, and decoding failures are not mistaken for EOF. Recover when the next record boundary is known, otherwise terminate after returning the error.

Generated model and query-engine consumers now propagate item errors. Simulator bridges temporarily collect streams until a follow-up introduces the fallible streaming interface in #520.

## Related Issues

Split off as a small incremental change from #520

_Written by Codex._

Authors:
  - Johan Peltenburg (https://github.com/johanpel)

Approvers:
  - Dhruv Vats (https://github.com/dhruv9vats)

URL: #537
Signed-off-by: Johan Peltenburg <johan.peltenburg+code@gmail.com>
Signed-off-by: Johan Peltenburg <johan.peltenburg+code@gmail.com>
@johanpel
johanpel force-pushed the schema-based-event-import-iterator-store branch from bbf4974 to e200c10 Compare August 6, 2026 10:22
Signed-off-by: Johan Peltenburg <johan.peltenburg+code@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant