Pass train_dtype into _load_model_memory_efficient#105
Conversation
Instead of extracting torch_dtype from base_kwargs inside the function, accept train_dtype as a proper parameter. This makes the API clearer and ensures the caller controls which dtype is used for model loading. Closes #34 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai>
|
Warning Review limit reached
More reviews will be available in 16 minutes and 44 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Summary
train_dtypeas an explicit parameter to_load_model_memory_efficientinstead of extracting it frombase_kwargsinternallytorch_dtypelookup and validation from inside the functionfrom_pretrainedto extract and validatetorch_dtypebefore passing it astrain_dtypetrain_dtypeoverridesbase_kwargs["torch_dtype"], and one verifying it works whenbase_kwargshas notorch_dtypeCloses #34
Test plan
ruff checkpassesruff format --checkpassestest_memory_efficient_loading_calls_alignment_hookupdated with new parametertest_memory_efficient_loading_uses_train_dtypeverifies override behaviortest_memory_efficient_loading_fallback_when_base_kwargs_missing_dtypeverifies no-dtype-in-kwargs caseCo-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com