Skip to content

feat(chat): generate replies and content in the user's language - #596

Open
abdulrafey1 wants to merge 3 commits into
refey/fix/lang-07-translate-keywordfrom
refey/feat/lang-08-chat-prompt
Open

feat(chat): generate replies and content in the user's language#596
abdulrafey1 wants to merge 3 commits into
refey/fix/lang-07-translate-keywordfrom
refey/feat/lang-08-chat-prompt

Conversation

@abdulrafey1

@abdulrafey1 abdulrafey1 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Part of: Sparkth UI, emails, API errors, and AI-generated content are English-only

Second of the phase-2 generation-language stack. Does not close the issue.

What

Makes the chat plugin write replies and generated course content in the user's stored preferred
language, instead of leaving the language to the model's guess.

Changes

  • feat(plugins): get_learning_design_system_prompt takes a resolved BCP 47 tag and names the
    output language explicitly in the system prompt
  • feat(plugins): resolve the tag once per request from the signed-in user's stored preference
  • fix(plugins): replace the ambiguous "Write in the user's language" instruction
  • test(plugins): prompt rendering per supported tag, a regression guard on the removed sentence,
    and route-level tests that the stored preference reaches the provider and re-resolves per request
  • docs(core): correct the config comment and the user guide, both of which said generated content
    ignores the preference

How to Test

  1. uv run pytest sparkth/plugins/chat/tests/test_prompt.py sparkth/plugins/chat/tests/test_completion_language.py -v
  2. PATCH /api/v1/user/me with {"language": "es"}, then start a chat and ask for a course
    outline in English — the reply and the outline come back in Spanish.
  3. Change the preference to fr mid-conversation and send another message: the next turn is in
    French, and earlier messages are left as they were.
  4. Clear the preference with {"language": null} and confirm replies fall back to
    DEFAULT_LANGUAGE.

Notes

No migration, no env var, no dependency.

The system prompt is rendered fresh on every request, so language re-resolves per turn. That is
intended: changing the setting applies from the next message onward and no conversation-level
pinning exists.

The template hands the model the refusal sentence and tells it to send that sentence verbatim, so
the new language directive carves it out explicitly. Without the carve-out the prompt contradicts
itself and the model's refusal would drift away from the deterministic refusal streamed on the
out-of-scope path.

Manual smoke test — not automatable, and the one thing the suite cannot cover. Every test here
mocks the LLM, so they prove the language reaches the prompt, not that the output is good:

  1. Generate a full course in es and in fr. Have a speaker check register and terminology, and
    confirm it does not read as translated English.
  2. Generate a course from an English source document into the target language — the highest-value
    case, and the one the directive's "regardless of the language of any uploaded source documents"
    clause exists for.
  3. Confirm assessment questions and their answer options and feedback are all translated.
    Partial translation of quiz options is a classic LLM failure.
  4. With a non-English preference set, ask explicitly "translate this into Spanish" (PR 1 unblocks
    that phrasing) and confirm the explicit request and the standing language directive interact
    sensibly rather than fighting.

This description was written with the assistance of an LLM (Claude).

abdulrafey1 and others added 3 commits August 14, 2026 00:01
get_learning_design_system_prompt now takes a resolved BCP 47 tag and names the language
explicitly in the prompt, covering replies, lesson text, assessment questions, answer
options and feedback. The completion handler resolves it once from the signed-in user's
stored preference.

Replaces the ambiguous "Write in the user's language", which read as the language the
user typed in rather than the one they configured, and would have fought the explicit
directive. The refusal sentence is carved out of the directive so the model reproduces it
verbatim and does not drift from the deterministic refusal streamed on the out-of-scope
path.

The prompt is rendered per request, so a preference changed mid-conversation applies from
the next turn. Corrects the config comment and the user guide, both of which stated that
generated content ignores the preference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_unset_preference_falls_back_to_the_platform_default only checked that
the default language's name appeared in the prompt. That assertion could
never fail: the template's OUTPUT LANGUAGE directive also names "English"
unconditionally, in every rendered prompt, regardless of the resolved
language. Add _other_supported_language_names(), mirrored from the later
MCP-prompt test, and assert none of those names leak into the prompt, so a
valid-but-wrong fallback resolution is actually caught.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comment above the language resolution call referred to a planning
task number that does not exist anywhere in the repository, and pointed
to title generation reusing the tag before that behavior lands on a
later branch. Remove the dangling reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@abdulrafey1 abdulrafey1 changed the title refey/feat/lang 08 chat prompt feat(chat): generate replies and content in the user's language Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants