Skip to content

feat(caches): Add PyIceberg-based cache for Apache Iceberg format #970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aaronsteers wants to merge 23 commits into
base: main
Choose a base branch
from
Open

feat(caches): Add PyIceberg-based cache for Apache Iceberg format #970

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6a88309
feat(caches): Add PyIceberg-based cache for Apache Iceberg format
devin-ai-integration[bot] Jan 29, 2026
56b3df1
fix: Address CodeRabbit feedback for IcebergCache
devin-ai-integration[bot] Jan 29, 2026
959dbdd
fix: Make _drop_temp_table a no-op to prevent data loss
devin-ai-integration[bot] Jan 29, 2026
76efd61
fix: Set type_converter_class on IcebergProcessor
devin-ai-integration[bot] Jan 29, 2026
0662720
feat: Add example script for IcebergCache with nested records
devin-ai-integration[bot] Jan 29, 2026
49f0397
test: Update test to reflect schema_name='main' for SQLite compatibility
devin-ai-integration[bot] Jan 29, 2026
4334764
fix: Add get_string_type and get_json_type methods to IcebergTypeConv…
devin-ai-integration[bot] Jan 29, 2026
5ad50af
fix: Fix loop counter variable shadowing in example script
devin-ai-integration[bot] Jan 29, 2026
04bc26c
feat(caches): Add sophisticated object typing configuration for Icebe…
devin-ai-integration[bot] Jan 29, 2026
cbe4aae
fix: Use consistent field ID counter to avoid collisions in Iceberg s…
devin-ai-integration[bot] Jan 29, 2026
aa2cc36
fix: Use as_json_strings mode in example script for compatibility
devin-ai-integration[bot] Jan 29, 2026
229c672
fix: Serialize schemaless arrays/objects in nested_types mode
devin-ai-integration[bot] Jan 29, 2026
affdc01
style: Fix ruff formatting
devin-ai-integration[bot] Jan 29, 2026
abc7edf
fix: Add timestamp conversion and improve recursive data converter
devin-ai-integration[bot] Jan 29, 2026
17fd26d
fix: Convert IcebergTypeConverter field ID counter to instance variable
devin-ai-integration[bot] Jan 29, 2026
31385bd
fix: Handle string boolean values correctly in convert_value_to_arrow…
devin-ai-integration[bot] Jan 29, 2026
b441ea0
fix: Remove unreachable None check in boolean handling
devin-ai-integration[bot] Jan 29, 2026
7d7df4a
fix: Reset field ID counter at start of schema generation
devin-ai-integration[bot] Jan 29, 2026
41d8a94
docs: Add experimental notice to Iceberg cache module and class docst…
devin-ai-integration[bot] Jan 29, 2026
710c792
docs: Add type handling configuration section to public module docstring
devin-ai-integration[bot] Jan 29, 2026
5eca9d6
tidying
aaronsteers Jan 30, 2026
6bd534c
feat: Implement merge-on-read using PyIceberg upsert for incremental …
devin-ai-integration[bot] Jan 31, 2026
40b2ab5
fix: Create directories for string local paths in _get_warehouse_path
devin-ai-integration[bot] Jan 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading