diff --git a/CHANGELOG.md b/CHANGELOG.md index 21eaea59..03e49ccf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,25 @@ +## Unreleased + +* **Potentially breaking behavior change:** native model cache defaults changed + without breaking Dart source compatibility. `DefaultModelDownloadManager()` now + prefers the platform shared cache on desktop/server instead of the process temp + directory, and mobile `DefaultModelDownloadManager.auto()` without an explicit + app-private directory now uses a best-effort temporary/cache fallback instead + of throwing. Apps or tests that asserted the old temp path or mobile exception + should pass an explicit cache directory or follow `MIGRATION.md`. + +* Added `DefaultModelDownloadManager.auto(...)` plus explicit model cache root + constructors for shared desktop caches, app-private mobile caches, + user-selected model libraries, and App Group containers. `auto(...)` now uses + platform-specific or generic app-private directories on Android/iOS when + supplied, and otherwise falls back to a best-effort temporary/cache directory + instead of requiring application `if` branches for simple cross-platform code. +* Updated the default native `DefaultModelDownloadManager()` constructor to use + the per-user shared model cache on desktop/server platforms and the mobile + app-private cache fallback, so plain `LlamaEngine(...)` remote source loads use + a platform-appropriate default while preserving a temporary fallback for hosts + that cannot expose a desktop cache environment. + ## 0.8.9 * Broadened the `hooks` dependency constraint to support both the existing @@ -22,12 +44,6 @@ the `llamadart_llama_cpp_flutter` Apple SwiftPM checksum, and aligned current README/website native override docs. -* Added `DefaultModelDownloadManager.auto(...)` plus explicit model cache root - constructors for shared desktop caches, app-private mobile caches, - user-selected model libraries, and App Group containers. Implicit shared cache - resolution now fails loudly on mobile and web where the OS cannot provide a - hidden cross-developer model folder. - ## 0.8.7 * Fixed multimodal chat-template rendering so templates that force-open diff --git a/MIGRATION.md b/MIGRATION.md index 486a092e..43912796 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -2,6 +2,83 @@ This document covers the major breaking upgrade paths. +## Next release: model download/cache defaults + +No source migration is required for existing calls: `DefaultModelDownloadManager` +constructors remain source-compatible, and the new mobile-specific directory +arguments on `DefaultModelDownloadManager.auto(...)` are optional. + +There are two intentional runtime default changes to be aware of: + +1. `DefaultModelDownloadManager()` no longer defaults to the process temporary + directory on desktop/server platforms. It now uses the same platform cache + root as `DefaultModelDownloadManager.auto()`: + + | Platform | New default root | + | --- | --- | + | Linux | `$XDG_CACHE_HOME/llamadart/models`, or `$HOME/.cache/llamadart/models` when `XDG_CACHE_HOME` is unset | + | macOS | `$HOME/Library/Caches/llamadart/models` | + | Windows | `%LOCALAPPDATA%\llamadart\models`, then `%APPDATA%\llamadart\models`, then `%USERPROFILE%\AppData\Local\llamadart\models` | + + If a desktop/server embedder cannot expose a home/cache environment, the + default constructor preserves compatibility by falling back to + `Directory.systemTemp/llamadart/models`. Explicit `auto(...)` and + `sharedCache(...)` calls still report cache-resolution errors so apps can + choose a durable directory. + +2. `DefaultModelDownloadManager.auto(platform: android/ios)` without an explicit + mobile directory no longer throws. It now uses an app-private temporary/cache + fallback at `Directory.systemTemp/llamadart/models`. This is convenient for + examples and rebuildable downloads, but large durable mobile model files + should still use an app-private cache/support directory resolved by the app. + For Flutter apps, prefer `path_provider.getApplicationCacheDirectory()` for + re-downloadable model caches; use `getApplicationSupportDirectory()` only for + app-owned durable support files when the app also accounts for platform + backup/no-backup policy. + +Recommended cross-platform setup: + +```dart +final engine = LlamaEngine( + LlamaBackend(), + modelDownloadManager: DefaultModelDownloadManager.auto( + // On Flutter, pass a path resolved by path_provider for the current app. + // For re-downloadable model caches, prefer getApplicationCacheDirectory(). + // Desktop/server ignores this and uses the per-user shared cache. + appPrivateCacheDirectory: appCacheModelsDirectory, + ), +); +``` + +If your app resolves platform-specific mobile directories ahead of time, pass +both without adding `Platform.isAndroid` / `Platform.isIOS` branches around the +download manager constructor: + +```dart +final manager = DefaultModelDownloadManager.auto( + androidAppPrivateCacheDirectory: androidModelsDirectory, + iosAppPrivateCacheDirectory: iosModelsDirectory, +); +``` + +To preserve the old temporary-cache behavior exactly on desktop/server, pass an +explicit directory: + +```dart +final manager = DefaultModelDownloadManager( + defaultCacheDirectory: path.join( + Directory.systemTemp.path, + 'llamadart', + 'models', + ), +); +``` + +If your application previously called `DefaultModelDownloadManager.auto()` on +Android/iOS and expected a `LlamaUnsupportedException`, update that test or call +`DefaultModelDownloadManager.sharedCache()` without `cacheDirectory` when you +specifically want to reject implicit mobile shared caches. + ## `0.6.3` -> `0.6.4` No public API break, but Android arm64 native packaging defaults changed. diff --git a/README.md b/README.md index e8e4947e..65448e05 100644 --- a/README.md +++ b/README.md @@ -280,10 +280,11 @@ import 'package:llamadart/llamadart.dart'; Future main() async { final String? appPrivateModelsDirectory = - await resolveMobileAppPrivateModelsDirectory(); + await resolveAppPrivateModelsDirectory(); // Desktop/server apps use a per-user shared cache. Android/iOS apps use the - // supplied app-private directory instead, so one code path works everywhere. + // supplied app-private directory when available, otherwise an app-private + // temporary/cache fallback. One constructor works across platforms. final engine = LlamaEngine( LlamaBackend(), modelDownloadManager: DefaultModelDownloadManager.auto( @@ -309,10 +310,16 @@ Future main() async { } ``` -`resolveMobileAppPrivateModelsDirectory()` represents your app storage layer, for -example a Flutter `path_provider` application-support path on Android/iOS. On -desktop/server, `auto(...)` ignores `appPrivateCacheDirectory` and uses the -per-user shared model cache. +`resolveAppPrivateModelsDirectory()` represents your app storage layer, for +example a Flutter `path_provider` application-support path. On desktop/server, +`auto(...)` ignores mobile app-private directory arguments and uses the per-user +shared model cache. On Android/iOS, pass `appPrivateCacheDirectory` when a storage +abstraction has already resolved the current platform's model directory, or pass +`androidAppPrivateCacheDirectory` and `iosAppPrivateCacheDirectory` when you want +one branch-free constructor call with platform-specific directories. If no mobile +directory is supplied, `auto(...)` falls back to the app-private system +temporary/cache directory; this is convenient for examples and rebuildable +downloads, but app-support storage is preferable for large durable model files. Native/file-backed backends stream remote models into the package-managed cache, resume partial `.part` downloads when the server supports HTTP Range and the @@ -326,24 +333,44 @@ and retries are rejected for local paths. `DefaultModelDownloadManager.auto(...)` is the recommended cross-platform entrypoint: desktop/server platforms use a per-user shared cache, while -Android/iOS use the app-private directory supplied by the app storage layer. +Android/iOS use a supplied platform-specific or generic app-private directory, +falling back to a best-effort system temporary/cache directory when omitted. +The default `DefaultModelDownloadManager()` constructor also uses the per-user +shared cache on desktop/server platforms and the same mobile app-private +temporary/cache fallback, so a plain `LlamaEngine(...)` has a platform-appropriate +default without application `if` branches. To preserve constructor compatibility +in unusual desktop/server embedders where no home/cache environment is available, +the default constructor falls back to the system temporary/cache root; +explicit desktop/server `auto(...)` and `sharedCache(...)` resolution still +reports cache-resolution errors so apps can choose a durable directory. `DefaultModelDownloadManager.sharedCache()` is the explicit desktop/server shared cache entrypoint, so multiple `llamadart` apps that use the same stable `ModelSource` can reuse one downloaded file. Mobile platforms do not have a safe implicit cross-developer model folder: use `DefaultModelDownloadManager.appPrivate(cacheDirectory: ...)` for normal -Android/iOS app storage, `userSelected(cacheDirectory: ...)` after an Android +Android/iOS app storage resolved by the app, `userSelected(cacheDirectory: ...)` after an Android Storage Access Framework-style user grant, or `appGroup(cacheDirectory: ...)` for explicitly configured iOS/macOS App Group containers. Web backends use origin-scoped browser/runtime caches instead of a file-backed shared directory. -Desktop shared-cache roots use the default `llamadart` namespace: +For Flutter apps, prefer `path_provider` over raw path guesses on mobile: +`getApplicationCacheDirectory()` is the closest match for re-downloadable model +caches, while `getApplicationSupportDirectory()` is appropriate only when the +app intentionally treats model files as durable support data and handles platform +backup/no-backup policy as needed. `getTemporaryDirectory()` also maps to +app-scoped cache locations such as Android `Context.getCacheDir` and Apple +`NSCachesDirectory`, but its contents may be cleared at any time. The +`Directory.systemTemp` fallback is therefore a compatibility fallback, not the +recommended durable mobile model-library location. + +Default cache roots use the default `llamadart` namespace: | Platform | Default path | | --- | --- | | Linux | `$XDG_CACHE_HOME/llamadart/models`, or `$HOME/.cache/llamadart/models` when `XDG_CACHE_HOME` is unset | | macOS | `$HOME/Library/Caches/llamadart/models` | | Windows | `%LOCALAPPDATA%\llamadart\models`, then `%APPDATA%\llamadart\models`, then `%USERPROFILE%\AppData\Local\llamadart\models` | +| Android/iOS | supplied app-private directory, preferably app cache/support resolved by the app, or `Directory.systemTemp/llamadart/models` as a best-effort cache fallback | Pass `namespace: 'your.namespace'` to `auto(...)` or `sharedCache(...)` to replace the `llamadart` path segment, or pass `cacheDirectory` to force an diff --git a/example/basic_app/README.md b/example/basic_app/README.md index 185b0e58..155cd2e1 100644 --- a/example/basic_app/README.md +++ b/example/basic_app/README.md @@ -7,6 +7,7 @@ A clean, organized CLI application demonstrating the capabilities of the `llamad - **Interactive Mode**: Have a back-and-forth conversation with an LLM in your terminal. - **Single Response Mode**: Pass a prompt as an argument for quick tasks. - **Automatic Model Management**: Automatically downloads models from Hugging Face if a URL is provided. +- **Platform Cache Defaults**: Remote model URLs use `DefaultModelDownloadManager`, so desktop/server runs share the per-user `llamadart` model cache while explicit local paths are loaded directly. - **Backend Optimization**: Defaults to GPU acceleration (Metal/Vulkan) when available. - **LoRA Adapters**: Load one or more LoRA adapters with repeated `--lora` flags. - **Structured Output**: Pass `--grammar` for GBNF-constrained generation. diff --git a/example/basic_app/lib/services/model_service.dart b/example/basic_app/lib/services/model_service.dart index 55997592..6364913e 100644 --- a/example/basic_app/lib/services/model_service.dart +++ b/example/basic_app/lib/services/model_service.dart @@ -1,86 +1,47 @@ import 'dart:io'; -import 'package:http/http.dart' as http; -import 'package:path/path.dart' as path; + +import 'package:llamadart/llamadart.dart'; /// Service for managing model downloads and local paths. class ModelService { - /// The directory where models are cached. - final String cacheDir; - /// Creates a model service with an optional [cacheDir]. ModelService([String? cacheDir]) - : cacheDir = cacheDir ?? path.join(Directory.current.path, 'models'); - - /// Ensures the model at [urlOrPath] is available locally. - /// If it's a URL, it downloads it. If it's a path, it verifies existence. - Future ensureModel(String urlOrPath) async { - if (urlOrPath.startsWith('http')) { - return await _downloadModel(urlOrPath); - } - - final file = File(urlOrPath); - if (!file.existsSync()) { - throw Exception('Model file not found at: $urlOrPath'); - } - return file; - } - - Future _downloadModel(String url) async { - final name = url.split('/').last.split('?').first; - final file = File(path.join(cacheDir, name)); + : _downloadManager = DefaultModelDownloadManager( + defaultCacheDirectory: cacheDir, + ); - if (file.existsSync() && file.lengthSync() > 0) { - return file; - } - - if (!file.parent.existsSync()) { - file.parent.createSync(recursive: true); - } + final DefaultModelDownloadManager _downloadManager; - print('Downloading model: $name'); - final client = http.Client(); - try { - final request = http.Request('GET', Uri.parse(url)); - final response = await client.send(request); + /// The directory where models are cached. + String get cacheDir => _downloadManager.defaultCacheDirectory; - if (response.statusCode != 200) { - throw Exception('Failed to download model: ${response.statusCode}'); + /// Ensures the model at [urlOrPath] is available locally. + /// If it's a URL, it downloads it through the package-managed model cache. + /// If it's a path, it verifies existence. + Future ensureModel(String urlOrPath) async { + final source = ModelSource.parse(urlOrPath); + if (source.isLocal) { + final file = File(source.path!); + if (!file.existsSync()) { + throw Exception('Model file not found at: ${source.path}'); } - - final contentLength = response.contentLength ?? 0; - var downloaded = 0; - final sink = file.openWrite(); - - await response.stream - .listen( - (chunk) { - sink.add(chunk); - downloaded += chunk.length; - if (contentLength > 0) { - final progress = (downloaded / contentLength * 100) - .toStringAsFixed(1); - stdout.write('\rProgress: $progress%'); - } else { - stdout.write( - '\rDownloaded: ${(downloaded / 1024 / 1024).toStringAsFixed(1)} MB', - ); - } - }, - onDone: () async { - await sink.close(); - print('\nDownload complete.'); - }, - onError: (e) { - sink.close(); - if (file.existsSync()) file.deleteSync(); - throw e; - }, - ) - .asFuture(); - return file; - } finally { - client.close(); } + + final entry = await _downloadManager.ensureModel( + source, + onProgress: (progress) { + final fraction = progress.fraction; + if (fraction != null) { + final percent = (fraction * 100).toStringAsFixed(1); + stdout.write('\rProgress: $percent%'); + } else { + final mb = (progress.receivedBytes / 1024 / 1024).toStringAsFixed(1); + stdout.write('\rDownloaded: $mb MB'); + } + }, + ); + stdout.writeln('\nDownload complete.'); + return File(entry.filePath); } } diff --git a/example/basic_app/pubspec.yaml b/example/basic_app/pubspec.yaml index 8e4e89b2..ec13e576 100644 --- a/example/basic_app/pubspec.yaml +++ b/example/basic_app/pubspec.yaml @@ -9,8 +9,6 @@ environment: dependencies: llamadart: path: ../.. # Use local version for testing - http: ^1.6.0 - path: ^1.8.3 args: ^2.4.2 sqlite3: ^3.1.6 sqlite_vector: ^0.9.85 diff --git a/example/chat_app/README.md b/example/chat_app/README.md index 32616b99..e65690bb 100644 --- a/example/chat_app/README.md +++ b/example/chat_app/README.md @@ -57,11 +57,14 @@ flutter test --run-skipped -t local-only \ ### 2. Choose and Download a Model 1. The app will open to a **Manage Models** screen. + - Native mobile/desktop builds store downloaded model files under the app's + application-specific cache `models` directory via `path_provider`; web builds use + browser Cache Storage/origin-scoped runtime caches. 2. Select one of the pre-configured models (for example: FunctionGemma 270M, Qwen3.5 0.8B/2B/4B/9B, Llama 3.2 3B, Gemma 3/3n, DeepSeek R1 distills). - Qwen3.5 presets now use Unsloth `Q4_K_M` GGUFs across platforms. - Quick picks: `0.8B` for web/older phones, `2B` for mobile + low-RAM laptops, `4B` for most native desktop/laptop runs, `9B` for desktop-class devices with more headroom. - Qwen3.5 small presets default to non-thinking mode for smoother latency and fewer reasoning loops; turn thinking on only when you need extra reasoning. -3. Tap the **Download** icon. The app uses `Dio` to download the model directly to your device's documents directory. +3. Tap the **Download** icon. The app uses `Dio` to download the model directly to your device's app-specific cache directory. 4. Once downloaded, tap **Select** to load the model. - Gemma 4 E2B is included as a GGUF + `mmproj` bundle. In the current `llama.cpp` mtmd path used here, that projector exposes vision support but diff --git a/example/chat_app/integration_test/smoke_test.dart b/example/chat_app/integration_test/smoke_test.dart index 29d9872c..448bfe5c 100644 --- a/example/chat_app/integration_test/smoke_test.dart +++ b/example/chat_app/integration_test/smoke_test.dart @@ -37,7 +37,7 @@ void main() { final Directory dataDir; if (Platform.isAndroid || Platform.isIOS) { - dataDir = await getApplicationDocumentsDirectory(); + dataDir = await getApplicationCacheDirectory(); } else { dataDir = Directory(path.join(Directory.current.path, 'models')); if (!dataDir.existsSync()) dataDir.createSync(recursive: true); diff --git a/example/chat_app/lib/services/model_service_io.dart b/example/chat_app/lib/services/model_service_io.dart index 021ef855..77354a77 100644 --- a/example/chat_app/lib/services/model_service_io.dart +++ b/example/chat_app/lib/services/model_service_io.dart @@ -32,7 +32,7 @@ class ModelServiceIO implements ModelService { @override Future getModelsDirectory() async { - final dir = await getApplicationDocumentsDirectory(); + final dir = await getApplicationCacheDirectory(); final modelsDir = Directory(p.join(dir.path, 'models')); if (!await modelsDir.exists()) { await modelsDir.create(recursive: true); diff --git a/lib/src/core/models/download/model_download_manager_stub.dart b/lib/src/core/models/download/model_download_manager_stub.dart index 7cc42240..1be67aba 100644 --- a/lib/src/core/models/download/model_download_manager_stub.dart +++ b/lib/src/core/models/download/model_download_manager_stub.dart @@ -11,6 +11,8 @@ class DefaultModelDownloadManager extends ThrowingModelDownloadManager { String namespace = 'llamadart', String? cacheDirectory, String? appPrivateCacheDirectory, + String? androidAppPrivateCacheDirectory, + String? iosAppPrivateCacheDirectory, ModelCachePlatform? platform, Map? environment, String? homeDirectory, diff --git a/lib/src/platform/io/model_download_manager_io.dart b/lib/src/platform/io/model_download_manager_io.dart index 76bd1581..ae415b12 100644 --- a/lib/src/platform/io/model_download_manager_io.dart +++ b/lib/src/platform/io/model_download_manager_io.dart @@ -16,28 +16,40 @@ const int _metadataSchemaVersion = 1; /// Native file-backed package-managed model download manager. class DefaultModelDownloadManager implements ModelDownloadManager { /// Creates a native model download/cache manager. + /// + /// When [defaultCacheDirectory] is omitted, desktop/server platforms use the + /// same per-user shared model cache as [DefaultModelDownloadManager.auto]. + /// Android and iOS use an app-private temporary/cache directory so constructing + /// the manager stays non-throwing; apps that need durable mobile storage should + /// pass an explicit directory or use [DefaultModelDownloadManager.appPrivate]. DefaultModelDownloadManager({String? defaultCacheDirectory}) : defaultCacheDirectory = - defaultCacheDirectory ?? - path.join(Directory.systemTemp.path, 'llamadart', 'models'); + defaultCacheDirectory ?? _defaultImplicitCacheDirectory(); /// Creates a manager using the recommended cache root for the current platform. /// /// Pass [cacheDirectory] to force a specific root on every platform, /// including an OS-granted mobile model library directory. Otherwise, /// desktop/server platforms use [defaultSharedCacheDirectory] with - /// [namespace], [environment], and [homeDirectory]. Android and iOS use - /// [appPrivateCacheDirectory] because `llamadart` cannot discover an app's - /// durable sandbox directory without app/platform storage APIs. + /// [namespace], [environment], and [homeDirectory]. Android and iOS use, in + /// order, their platform-specific app-private directory argument, + /// [appPrivateCacheDirectory], or an app-private temporary/cache fallback. + /// + /// Use [androidAppPrivateCacheDirectory] and [iosAppPrivateCacheDirectory] + /// when an app resolves both platform directories up front and wants one + /// branch-free `auto(...)` call. Use [appPrivateCacheDirectory] when a storage + /// abstraction such as Flutter `path_provider` has already returned the + /// current platform's app-private model directory. /// - /// Throws [LlamaUnsupportedException] on Android or iOS when neither - /// [cacheDirectory] nor [appPrivateCacheDirectory] is supplied, on web because - /// browser caches are origin-scoped instead of file-backed, and on unknown - /// platforms where no implicit shared cache root is known. + /// Throws [LlamaUnsupportedException] on web because browser caches are + /// origin-scoped instead of file-backed, and on unknown platforms where no + /// implicit cache root is known. factory DefaultModelDownloadManager.auto({ String namespace = 'llamadart', String? cacheDirectory, String? appPrivateCacheDirectory, + String? androidAppPrivateCacheDirectory, + String? iosAppPrivateCacheDirectory, ModelCachePlatform? platform, Map? environment, String? homeDirectory, @@ -61,17 +73,18 @@ class DefaultModelDownloadManager implements ModelDownloadManager { } if (effectivePlatform.isMobile) { - final appPrivateDirectory = _nonEmpty(appPrivateCacheDirectory); - if (appPrivateDirectory != null) { - return DefaultModelDownloadManager.appPrivate( - cacheDirectory: appPrivateDirectory, - ); - } - throw LlamaUnsupportedException( - 'DefaultModelDownloadManager.auto cannot choose a durable model cache ' - 'directory for ${effectivePlatform.name}. Pass appPrivateCacheDirectory ' - 'from your app storage layer, or pass cacheDirectory for an explicit ' - 'OS-granted model library.', + return DefaultModelDownloadManager.appPrivate( + cacheDirectory: + _mobileAppPrivateCacheDirectory( + effectivePlatform, + appPrivateCacheDirectory: appPrivateCacheDirectory, + androidAppPrivateCacheDirectory: androidAppPrivateCacheDirectory, + iosAppPrivateCacheDirectory: iosAppPrivateCacheDirectory, + ) ?? + _defaultImplicitCacheDirectoryFor( + effectivePlatform, + namespace: namespace, + ), ); } @@ -120,11 +133,12 @@ class DefaultModelDownloadManager implements ModelDownloadManager { ); } - /// Creates a manager rooted at an app-private durable cache directory. + /// Creates a manager rooted at an app-private model cache directory. /// - /// Flutter apps typically pass an application-support or documents directory - /// resolved with platform storage APIs. This avoids cross-app storage claims on - /// mobile while still deduping model downloads within the app. + /// Flutter apps typically pass an application cache directory for + /// re-downloadable models, or an application-support directory when the app + /// owns the platform backup/no-backup policy. This avoids cross-app storage + /// claims on mobile while still deduping model downloads within the app. factory DefaultModelDownloadManager.appPrivate({ required String cacheDirectory, }) { @@ -1454,6 +1468,54 @@ String? _nonEmpty(String? value) { return value; } +String _defaultImplicitCacheDirectory() { + final platform = ModelCachePlatform.parse(Platform.operatingSystem); + try { + return _defaultImplicitCacheDirectoryFor(platform); + } on LlamaUnsupportedException { + // Keep the default constructor non-throwing for compatibility with older + // embedders and unusual host environments that do not expose a home/cache + // environment. Explicit auto/sharedCache calls still surface the actionable + // resolution error. + return _temporaryCacheDirectory(); + } +} + +String _defaultImplicitCacheDirectoryFor( + ModelCachePlatform platform, { + String namespace = 'llamadart', +}) { + if (platform.supportsImplicitSharedModelCache) { + return DefaultModelDownloadManager.defaultSharedCacheDirectory( + platform: platform, + namespace: namespace, + ); + } + return _temporaryCacheDirectory(namespace: namespace); +} + +String _temporaryCacheDirectory({String namespace = 'llamadart'}) { + return path.join( + Directory.systemTemp.path, + _validateCacheNamespace(namespace), + 'models', + ); +} + +String? _mobileAppPrivateCacheDirectory( + ModelCachePlatform platform, { + required String? appPrivateCacheDirectory, + required String? androidAppPrivateCacheDirectory, + required String? iosAppPrivateCacheDirectory, +}) { + final platformPrivateDirectory = switch (platform) { + ModelCachePlatform.android => _nonEmpty(androidAppPrivateCacheDirectory), + ModelCachePlatform.ios => _nonEmpty(iosAppPrivateCacheDirectory), + _ => null, + }; + return platformPrivateDirectory ?? _nonEmpty(appPrivateCacheDirectory); +} + String _validateCacheNamespace(String namespace) { final trimmed = namespace.trim(); if (!RegExp(r'^[A-Za-z0-9][A-Za-z0-9._-]*$').hasMatch(trimmed)) { diff --git a/test/unit/core/models/download/model_download_manager_stub_test.dart b/test/unit/core/models/download/model_download_manager_stub_test.dart index 7fbb241f..614de300 100644 --- a/test/unit/core/models/download/model_download_manager_stub_test.dart +++ b/test/unit/core/models/download/model_download_manager_stub_test.dart @@ -20,6 +20,8 @@ void main() { test('auto constructor compiles on non-IO platforms', () async { const manager = DefaultModelDownloadManager.auto( appPrivateCacheDirectory: '/app/models', + androidAppPrivateCacheDirectory: '/android/models', + iosAppPrivateCacheDirectory: '/ios/models', ); await expectLater( @@ -31,6 +33,8 @@ void main() { test('default cache directory getter throws unsupported exception', () { const manager = DefaultModelDownloadManager.auto( appPrivateCacheDirectory: '/app/models', + androidAppPrivateCacheDirectory: '/android/models', + iosAppPrivateCacheDirectory: '/ios/models', ); expect( diff --git a/test/unit/platform/io/model_download_manager_io_test.dart b/test/unit/platform/io/model_download_manager_io_test.dart index b3125f3a..ce55eec7 100644 --- a/test/unit/platform/io/model_download_manager_io_test.dart +++ b/test/unit/platform/io/model_download_manager_io_test.dart @@ -70,6 +70,25 @@ void main() { ); }); + test('constructor defaults to the platform implicit cache directory', () { + final manager = DefaultModelDownloadManager(); + final platform = ModelCachePlatform.parse(Platform.operatingSystem); + + if (platform.supportsImplicitSharedModelCache) { + expect( + manager.defaultCacheDirectory, + DefaultModelDownloadManager.defaultSharedCacheDirectory( + platform: platform, + ), + ); + } else { + expect( + manager.defaultCacheDirectory, + path.join(Directory.systemTemp.path, 'llamadart', 'models'), + ); + } + }); + test('auto uses desktop shared cache directories', () { final linuxManager = DefaultModelDownloadManager.auto( platform: ModelCachePlatform.linux, @@ -94,7 +113,7 @@ void main() { ); }); - test('auto uses explicit app-private directories on mobile', () { + test('auto uses app-private directories on mobile', () { final androidManager = DefaultModelDownloadManager.auto( platform: ModelCachePlatform.android, appPrivateCacheDirectory: path.join(tempDir.path, 'android-private'), @@ -108,6 +127,40 @@ void main() { expect(iosManager.defaultCacheDirectory, endsWith('ios-private')); }); + test('auto uses platform-specific app-private directories on mobile', () { + final commonPrivate = path.join(tempDir.path, 'common-private'); + final androidPrivate = path.join(tempDir.path, 'android-private'); + final iosPrivate = path.join(tempDir.path, 'ios-private'); + + final androidManager = DefaultModelDownloadManager.auto( + platform: ModelCachePlatform.android, + appPrivateCacheDirectory: commonPrivate, + androidAppPrivateCacheDirectory: androidPrivate, + iosAppPrivateCacheDirectory: iosPrivate, + ); + final iosManager = DefaultModelDownloadManager.auto( + platform: ModelCachePlatform.ios, + appPrivateCacheDirectory: commonPrivate, + androidAppPrivateCacheDirectory: androidPrivate, + iosAppPrivateCacheDirectory: iosPrivate, + ); + final macosManager = DefaultModelDownloadManager.auto( + platform: ModelCachePlatform.macos, + environment: const {}, + homeDirectory: '/Users/alice', + appPrivateCacheDirectory: commonPrivate, + androidAppPrivateCacheDirectory: androidPrivate, + iosAppPrivateCacheDirectory: iosPrivate, + ); + + expect(androidManager.defaultCacheDirectory, androidPrivate); + expect(iosManager.defaultCacheDirectory, iosPrivate); + expect( + macosManager.defaultCacheDirectory, + '/Users/alice/Library/Caches/llamadart/models', + ); + }); + test('auto explicit cache directory overrides platform defaults', () { final manager = DefaultModelDownloadManager.auto( platform: ModelCachePlatform.android, @@ -117,25 +170,71 @@ void main() { expect(manager.defaultCacheDirectory, endsWith('user-library')); }); - test('auto rejects mobile without an explicit durable directory', () { + test('auto falls back to app-private temp cache on mobile', () { for (final platform in [ ModelCachePlatform.android, ModelCachePlatform.ios, ]) { + final manager = DefaultModelDownloadManager.auto(platform: platform); + expect( - () => DefaultModelDownloadManager.auto(platform: platform), - throwsA( - isA().having( - (error) => error.toString(), - 'message', - contains('appPrivateCacheDirectory'), - ), - ), + manager.defaultCacheDirectory, + path.join(Directory.systemTemp.path, 'llamadart', 'models'), reason: platform.name, ); } }); + test('auto applies namespace to mobile temp cache fallback', () { + final androidManager = DefaultModelDownloadManager.auto( + namespace: 'com.example.app', + platform: ModelCachePlatform.android, + ); + final iosManager = DefaultModelDownloadManager.auto( + namespace: 'com.example.app', + platform: ModelCachePlatform.ios, + ); + + expect( + androidManager.defaultCacheDirectory, + path.join(Directory.systemTemp.path, 'com.example.app', 'models'), + ); + expect( + iosManager.defaultCacheDirectory, + path.join(Directory.systemTemp.path, 'com.example.app', 'models'), + ); + }); + + test('auto validates namespace for mobile temp cache fallback', () { + expect( + () => DefaultModelDownloadManager.auto( + namespace: '../bad', + platform: ModelCachePlatform.android, + ), + throwsArgumentError, + ); + }); + + test( + 'auto uses generic mobile directory when specific directory is blank', + () { + final genericPrivate = path.join(tempDir.path, 'generic-private'); + final androidManager = DefaultModelDownloadManager.auto( + platform: ModelCachePlatform.android, + appPrivateCacheDirectory: genericPrivate, + androidAppPrivateCacheDirectory: ' ', + ); + final iosManager = DefaultModelDownloadManager.auto( + platform: ModelCachePlatform.ios, + appPrivateCacheDirectory: genericPrivate, + iosAppPrivateCacheDirectory: '', + ); + + expect(androidManager.defaultCacheDirectory, genericPrivate); + expect(iosManager.defaultCacheDirectory, genericPrivate); + }, + ); + test('auto rejects non-file-backed and unknown platforms', () { for (final platform in [ ModelCachePlatform.web, diff --git a/website/docs/changelog/recent-releases.md b/website/docs/changelog/recent-releases.md index 70d7281a..5de89dcc 100644 --- a/website/docs/changelog/recent-releases.md +++ b/website/docs/changelog/recent-releases.md @@ -7,6 +7,28 @@ For canonical full release notes, use: - [`CHANGELOG.md`](https://github.com/leehack/llamadart/blob/main/CHANGELOG.md) +## Unreleased + +- **Potentially breaking behavior change:** native model cache defaults changed + without breaking Dart source compatibility. `DefaultModelDownloadManager()` now + prefers the platform shared cache on desktop/server instead of the process temp + directory, and mobile `DefaultModelDownloadManager.auto()` without an explicit + app-private directory now uses a best-effort temporary/cache fallback instead + of throwing. Apps or tests that asserted the old temp path or mobile exception + should pass an explicit cache directory or follow `MIGRATION.md`. + +- Added `DefaultModelDownloadManager.auto(...)` plus explicit model cache root + constructors for shared desktop caches, app-private mobile caches, + user-selected model libraries, and App Group containers. `auto(...)` now uses + platform-specific or generic app-private directories on Android/iOS when + supplied, and otherwise falls back to a best-effort temporary/cache directory + instead of requiring application `if` branches for simple cross-platform code. +- Updated the default native `DefaultModelDownloadManager()` constructor to use + the per-user shared model cache on desktop/server platforms and the mobile + app-private cache fallback, so plain `LlamaEngine(...)` remote source loads use + a platform-appropriate default while preserving a temporary fallback for hosts + that cannot expose a desktop cache environment. + ## 0.8.9 - Broadened the `hooks` dependency constraint to support both the existing @@ -31,12 +53,6 @@ For canonical full release notes, use: refreshed the `llamadart_llama_cpp_flutter` Apple SwiftPM checksum, and aligned current README/website native override docs. -- Added `DefaultModelDownloadManager.auto(...)` plus explicit model cache root - constructors for shared desktop caches, app-private mobile caches, - user-selected model libraries, and App Group containers. Implicit shared cache - resolution now fails loudly on mobile and web where the OS cannot provide a - hidden cross-developer model folder. - ## 0.8.7 - Fixed multimodal chat-template rendering so templates that force-open diff --git a/website/docs/guides/model-lifecycle.md b/website/docs/guides/model-lifecycle.md index 5e1d5cf5..eaf88e9a 100644 --- a/website/docs/guides/model-lifecycle.md +++ b/website/docs/guides/model-lifecycle.md @@ -171,7 +171,7 @@ final cancelToken = ModelDownloadCancelToken(); // Desktop/server uses the shared cache. Android/iOS uses the supplied // app-private directory, so the call site can stay platform-neutral. final manager = DefaultModelDownloadManager.auto( - appPrivateCacheDirectory: appSupportModelsDirectory, + appPrivateCacheDirectory: appCacheModelsDirectory, ); final engine = LlamaEngine(LlamaBackend(), modelDownloadManager: manager); @@ -208,15 +208,15 @@ Recommended cache roots: ```dart // Cross-platform default: desktop shared cache, mobile app-private cache. final crossPlatformManager = DefaultModelDownloadManager.auto( - appPrivateCacheDirectory: appSupportModelsDirectory, + appPrivateCacheDirectory: appCacheModelsDirectory, ); // Desktop/server: per-user cache, shared across llamadart apps using the same source. final desktopManager = DefaultModelDownloadManager.sharedCache(); -// Mobile default: app-private durable storage resolved by your app/platform layer. +// Mobile default: app-private cache/support storage resolved by your app/platform layer. final mobileManager = DefaultModelDownloadManager.appPrivate( - cacheDirectory: appSupportModelsDirectory, + cacheDirectory: appCacheModelsDirectory, ); // Android cross-developer sharing: only after the user grants this directory. @@ -231,23 +231,41 @@ final appGroupLibrary = DefaultModelDownloadManager.appGroup( ``` `auto(...)` is the easiest way to keep one code path: desktop/server platforms -use the shared cache, and Android/iOS use `appPrivateCacheDirectory`. If that -mobile directory is missing, `auto(...)` throws an actionable -`LlamaUnsupportedException` instead of falling back to a temporary or hidden -cross-app location. `sharedCache()` intentionally refuses to invent a mobile -shared folder when no `cacheDirectory` is supplied. Android requires an explicit -user or app-managed grant for a shared model library, and iOS requires an App -Group for same-group apps. For apps from unrelated developers, iOS can load -user-picked files but does not provide a writable hidden shared model cache. Web -model caches remain origin-scoped browser/runtime caches. - -Desktop shared-cache roots use the default `llamadart` namespace: +use the shared cache, and Android/iOS use a supplied platform-specific or generic +app-private directory. If no mobile directory is supplied, `auto(...)` falls back +to a best-effort system temporary/cache directory; it never invents a hidden +cross-developer shared folder. `sharedCache()` intentionally refuses to invent a +mobile shared folder when no `cacheDirectory` is supplied. Android requires an +explicit user or app-managed grant for a shared model library, and iOS requires +an App Group for same-group apps. For apps from unrelated developers, iOS can +load user-picked files but does not provide a writable hidden shared model cache. +Web model caches remain origin-scoped browser/runtime caches. + +The default `DefaultModelDownloadManager()` constructor follows the same desktop +and mobile defaults. To preserve constructor compatibility in unusual desktop or +server embedders where no home/cache environment is available, the default +constructor falls back to the system temporary/cache root. Explicit +desktop/server `auto(...)` and `sharedCache(...)` resolution still reports +cache-resolution errors so apps can choose a durable directory. + +On Flutter mobile apps, resolve app-private paths with `path_provider` rather +than hard-coded platform paths. Use `getApplicationCacheDirectory()` for +re-downloadable model caches; use `getApplicationSupportDirectory()` only when +the app intentionally owns durable support files and accounts for platform +backup/no-backup policy. `getTemporaryDirectory()` is also app-scoped and maps to +platform cache APIs such as Android `Context.getCacheDir` and Apple +`NSCachesDirectory`, but those files may be cleared. The `Directory.systemTemp` +fallback is a compatibility path for simple examples and embedders, not the +recommended durable mobile model-library location. + +Default cache roots use the default `llamadart` namespace: | Platform | Default path | | --- | --- | | Linux | `$XDG_CACHE_HOME/llamadart/models`, or `$HOME/.cache/llamadart/models` when `XDG_CACHE_HOME` is unset | | macOS | `$HOME/Library/Caches/llamadart/models` | | Windows | `%LOCALAPPDATA%\llamadart\models`, then `%APPDATA%\llamadart\models`, then `%USERPROFILE%\AppData\Local\llamadart\models` | +| Android/iOS | supplied app-private directory, preferably app cache/support resolved by the app, or `Directory.systemTemp/llamadart/models` as a best-effort cache fallback | Pass `namespace: 'your.namespace'` to `auto(...)` or `sharedCache(...)` to replace the `llamadart` path segment, or pass `cacheDirectory` to force an @@ -257,7 +275,7 @@ The download manager can also inspect and clean the persisted cache: ```dart final manager = DefaultModelDownloadManager.auto( - appPrivateCacheDirectory: appSupportModelsDirectory, + appPrivateCacheDirectory: appCacheModelsDirectory, ); final cached = await manager.list(); @@ -279,7 +297,7 @@ that lifecycle without depending on Flutter: ```dart final controller = ModelDownloadController( manager: DefaultModelDownloadManager.auto( - appPrivateCacheDirectory: appSupportModelsDirectory, + appPrivateCacheDirectory: appCacheModelsDirectory, ), ); @@ -370,11 +388,16 @@ strings, fragments, and userinfo are redacted from display strings and metadata. ### Mobile large-download guidance -For Flutter apps on Android/iOS, pass an app-controlled `cacheDirectory` from -your storage strategy (for example an application-support or documents directory -selected by `path_provider`) to `DefaultModelDownloadManager.appPrivate(...)`, -or pass it as `appPrivateCacheDirectory` to `DefaultModelDownloadManager.auto(...)` -when you want one constructor across desktop and mobile. +For Flutter apps on Android/iOS, prefer an app-controlled directory from your +storage strategy. Use `path_provider.getApplicationCacheDirectory()` for +re-downloadable model caches, or `getApplicationSupportDirectory()` only when +the app owns the platform backup/no-backup policy. Pass the current platform's +resolved path as `appPrivateCacheDirectory`, or pass both +`androidAppPrivateCacheDirectory` and `iosAppPrivateCacheDirectory` if your app +resolves them up front and wants one branch-free `DefaultModelDownloadManager.auto(...)` +call. If omitted, `auto(...)` still uses an app-private temporary/cache fallback, +which is convenient for rebuildable downloads but less explicit than a directory +resolved by your app. Surface progress/cancel controls in the UI, keep downloads serialized for large GGUF files, and tell users to keep the app open for the foreground download. Do not cancel purely because the app receives a lifecycle pause: Android/iOS may