webshart: custom caption key(s) for multicaption support - #3208
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
It changes caption resolution and cache-file keying in the core training data pipeline and relies on external webshart reader APIs (open_shard/read_sample_json) that cannot be verified from this checkout, so human sign-off is warranted despite strong test coverage.
Pull request overview
This PR adds a webshart.caption_key dataset option that lets users select which caption field(s) to extract from Webshart metadata, supporting a single key or an ordered list of keys for multicaption training. The feature threads through configuration parsing/serialization, validation, the backend builder, both the data and metadata backends, the WebUI dataset templates, and documentation in all supported languages. Selected captions are returned as variants (not concatenated), with sidecar JSON reads on demand and cache files partitioned by a hash of the selected keys so different selections don't collide.
Changes:
- New
caption_keyoption with validation, config round-trip, builder pass-through, and backend resolution via_select_caption_keys(priority: JSON metadata → indexed metadata →captionsdicts). - WebUI textarea (one key per line) plus store logic to persist single-string vs list values; cache/metadata files suffixed with a
sha256digest of the key selection. - Documentation (
DATALOADER.md+ zh/pt-BR/ja/hi/es) and comprehensive unit, integration, and E2E tests.
File summaries
| File | Description |
|---|---|
| simpletuner/helpers/data_backend/webshart.py | Adds caption_key param, _select_caption_keys, sidecar JSON read, and instance-representation round-trip |
| simpletuner/helpers/metadata/backends/webshart.py | Digest-based cache-file separation, caption override in _metadata_for_entry, and skip logic when keys are set |
| simpletuner/helpers/data_backend/config/validators.py | New validate_webshart_caption_key validation |
| simpletuner/helpers/data_backend/config/image.py | Config field, parsing, and serialization for caption_key |
| simpletuner/helpers/data_backend/builders/webshart.py | Passes caption_key from config to the backend |
| simpletuner/templates/components/dataloader/webshart_caption_key.html | New textarea component for caption keys |
| simpletuner/templates/components/dataloader/sections/storage.html, storage_body.html | Include the new component in inline/modal views; extends param filter |
| simpletuner/static/js/dataloader-section-component.js | setWebshartCaptionKeys normalizes textarea input to string/list |
| simpletuner/templates/trainer_htmx.html | Serializes caption_key on save |
| documentation/DATALOADER.md (+ zh/pt-BR/ja/hi/es) | Documents the new option in all languages |
| tests/test_webshart_backend.py, tests/test_webui_e2e.py | Unit, integration, and E2E coverage |
Review details
- Files reviewed: 18/18 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This pull request introduces support for selecting specific caption fields from Webshart datasets using a new
caption_keyoption. The change is implemented across the data backend, configuration, and documentation in multiple languages. The new feature allows users to specify which caption fields to use, supporting both single and multiple keys, with robust validation and handling in the backend code. Documentation has been updated to explain the new option and its behavior in detail.Feature: Webshart caption key selection
webshart.caption_keyoption to the configuration (ImageBackendConfig, config parsing, and serialization) to allow users to specify one or more caption fields to extract from Webshart metadata. This includes validation to ensure the key(s) are non-empty strings or lists of non-empty strings. [1] [2] [3] [4]WebshartDataBackendto accept and use thecaption_keyparameter. The backend now extracts the specified caption fields in priority order, normalizes them, and returns them as variants (not concatenated), with correct fallback and sidecar reading logic. [1] [2] [3] [4]caption_keysetting is included in backend instance representations and restored correctly from serialization. [1] [2]caption_keyfrom config to the backend.Documentation updates
webshart.caption_keyoption, its usage, lookup order, and effects on caching and sample selection. [1] [2] [3] [4] [5] [6]Other minor changes
sha256import insimpletuner/helpers/metadata/backends/webshart.pyCI regression: Webshart 0.5.3 discards unknown fields placed directly on an index entry. The failing fixture depended on unreleased Webshart support for those fields. The fixtures now store literal custom keys in
json_metadataand its nestedcaptionsdictionary, and exercise direct index lookup using the supportedcaptionsfield. The dataloader documentation and all translations clarify this storage requirement.Validation: reproduced the original assertion failure using the published Webshart 0.5.3 wheel, then passed all 105 tests in
tests.test_webshart_backend,tests.test_backend_builders,tests.test_prompts, andtests.test_prompthandleragainst that same isolated wheel. Black, isort, and the staged diff check passed.