[Notebook Migration WIP] Adding R support

[zyratlo]

This post is to let the community know what is planned for R support in the notebook migration tool (#4301). The Jupyter notebook path is already merged. Python file input (#8007) is merged, and Python folder input (#8008) is in review and should merge soon. R support is new and has not been started. The tracking issue is #8009.

Where things stand
The tool assumes Python throughout. The prompts teach the model Texera's Python UDF interface, and every generated operator is built from the PythonUDFV2 schema. Texera already has R UDF operators, so the target exists, the tool just never produces it.

What is planned

• An R prompt set that teaches the model Texera's R UDF interface the way the current prompts teach the Python one. Python migrations keep their exact prompt text and are unaffected.
• There are already different tabs in the tool for notebooks, Python files, and Python folders. R gets its own tab
• R supports the same three input types as Python: notebook, single file, and folder. File and folder input reuse the segmentation and assembly logic from [Notebook Migration] Support migrating Python files #8007 and [Notebook Migration] Support migrating Python folders from local files and GitHub #8008.

The upload path, cell to operator mapping, Jupyter panel, workflow assembly, and delete path do not depend on the language and carry over unchanged.

Open questions

• Mixed input: A folder can contain both R and Python. The simplest first version rejects mixed folders. The alternative is one workflow with operators in both languages.
• The Jupyter panel: The panel is read only, so displaying an R notebook should not need an R kernel. If we later allow running cells from the panel, the Jupyter deployment will need an R kernel added.

[sshiv012]

I like the direction of starting with the simpler version first. Reusing the existing migration pipeline and only making the prompt/operator construction language-specific seems like the right boundary.
For mixed folders, I’d prefer rejecting mixed R/Python input in the first version, but I think we should design the implementation so language is a parameter rather than adding R-specific branches throughout the tool.
For the Jupyter panel, I also agree that read-only R notebook display should not depend on having an R kernel. Kernel support feels like a separate capability that we should only introduce once we actually support executing cells from the panel.
One thing I’d be interested in is how language is detected for notebooks vs files/folders. For files the extension is straightforward, but for notebooks I think kernel/language metadata should probably be the default signal, with a fallback if metadata is missing or ambiguous?

[zyratlo]

I agree on language as a parameter, that's the plan.

For detection, the tool doesn't really detect anything itself since the import modal has a separate tab for each input type (notebook, Python file, Python folder), and R will get its own tabs alongside them. You can see the tabs in the demo video in #8767. So the user tells us the language and input type up front by picking the tab, and the tab's file picker will only accept the matching file extension.

After that we run checks on what was uploaded. Right now they are structural: the extension has to match, a notebook has to parse with a cells array, a script can't be empty, and a folder has to contain source files and stay under the size caps. For R I'd probably add a language check on top, specifically for notebooks since those can contain both R and Python (like your metadata point). If the metadata disagrees with the tab (ex. a Python notebook in the R tab) I'd reject with a message. If it's missing entirely I'd trust the user-selected tab.

Does trusting the tab when metadata is missing seem reasonable to you, or do you think its better to fail there too?

[sshiv012]

I think trusting the selected tab when notebook metadata is missing is ok. The tab is already an explicit user declaration of intent, so I’d treat metadata as an extra signal, not the primary source of truth. I’d prefer what you suggested over failing on missing metadata because notebook metadata is not always reliable or complete, and rejecting otherwise valid notebooks would add friction without really improving correctness. The only thing I’d consider is making the error wording clear that it’s a mismatch rather than saying the notebook itself is invalid, like maybe “This notebook appears to use Python, but you selected R notebook import.”

[zyratlo]

The error wording for the new R support makes it clear to the user in the error if they upload a Python notebook in the R notebook section. However, during my investigation I realized that some of my previous implementations still give generic error messages, so thank you for the catch! I've raised it as #8929 with more specific details if you are interested.

[sshiv012]

Thank you Ryan, yes I'll check this out!