Skip to content

Fix ROSE As a Service merge conflicts#88

Merged
AymenFJA merged 44 commits into
prototype/raasfrom
fix/raas-main-merge-conflicts
Mar 25, 2026
Merged

Fix ROSE As a Service merge conflicts#88
AymenFJA merged 44 commits into
prototype/raasfrom
fix/raas-main-merge-conflicts

Conversation

@AymenFJA

@AymenFJA AymenFJA commented Mar 16, 2026

Copy link
Copy Markdown
Collaborator

Update the raas branch with the core changes on the API level

AymenFJA and others added 30 commits February 26, 2026 17:04
1-Migrate the runtime system used by ROSE from asyncflow to RHAPSODY
2-Update tests to comply with the changes above
3-Update examples/tutorials and use cases to comply with the changes above
1-Reorder the exmples
2-Sort and isoalte the tutorials
3-Update README with changes
1-Make the ParallelLearner API emits real time results per iteration like other learners
2-Update tests
3-Update examples
…learners

- Pass learner_configs[learner_id] directly to start(), removing the sequential_config intermediate variable
- Add factory comment explaining the closure-in-loop pitfall
- Fix print typo: ActiveLearner-{id}] → [ActiveLearner-{id}]
- Apply the same changes to reinforcement_learner
- Remove sequential_config and add factory comment in uq_active_learner
- Update unit test assertion to match corrected print format
…learner

Parallel Learner API emits async results
1-Base Tracking capabilites
2-PNP approach for trackers
3-Native File based exp. tracker
4-MLFLOW exp. tracker
5-ClearML exp. tracker
6-Extend examples
7-Extend docs
1-Test suite for tracking
2-Examples for tracking
3-Update docs
1-Modify the approach so it aligns with ROSE working on OUTER loop of the learning and not the INNER Loop
2-Update Docs
3-Update Examples
4-Update Tests
1-Isolate the track parameters from the decor kwargs and uses log_params
2-Fix examples to be more realistic
3-Update tests
4-Test with MLFlow and ClearML UI
1-Fix 1 — ClearML series names: ClearMLTracker adds an optional learner_names parameter, improves series naming, and fixes a bug where learner_id=0 was incorrectly mapped to value.
2-Fix 2 — Non-numeric config logging: Add string config logging for MLflow (via tags) and ClearML (via connected hyperparameters) and move _TASK_NAMES to module level to avoid recreating it each iteration.
…cking

Adding base, MLFlow, ClearML experiment tracking
1-Fix pre-commits
2-Sets the RAAS branch up to data with main new changes
@AymenFJA AymenFJA self-assigned this Mar 16, 2026
@AymenFJA AymenFJA added High raas ROSE As A Service Q1 labels Mar 16, 2026
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the core learner APIs, particularly for parallel execution, to provide a more consistent and real-time observability experience. It introduces a robust, pluggable tracking system with out-of-the-box integrations for MLflow and ClearML, enhancing experiment management and reproducibility. Concurrently, it updates key dependencies, modernizes Python version support, and integrates comprehensive code quality checks into the development workflow.

Highlights

  • Unified Async-Iterator API for Parallel Learners: Parallel learners (ParallelActiveLearner, ParallelReinforcementLearner, ParallelUQLearner) now expose a unified async for state in learner.start(): interface, streaming IterationState objects in real-time as iterations complete. The teach() and learn() methods are deprecated in favor of this new API.
  • Enhanced Observability and Tracking Integrations: A new pluggable tracking system (TrackerBase) has been introduced, allowing seamless integration with experiment tracking platforms. Dedicated MLflowTracker and ClearMLTracker implementations are provided, automatically logging pipeline manifests, per-iteration metrics, and stop reasons without requiring tracking code within the main loop. A simple HPC_FileTracker is also available for robust logging in HPC environments.
  • Dependency and Python Version Updates: Core execution backends have been migrated from radical.asyncflow to rhapsody-py, improving modularity and future compatibility. The minimum supported Python version has been raised to 3.10, and Python 3.9 support has been dropped across tooling and CI.
  • Improved Code Quality and CI: A comprehensive .pre-commit-config.yaml has been added, integrating docformatter, ruff, actionlint, and other standard code quality checks. These pre-commit hooks are now enforced as a required CI gate, ensuring consistent code style and preventing common issues.
  • New Tutorials and Documentation: Two new tutorials, 03-highly-parallel-surrogates and 04-al-algorithm-selector, have been added to showcase advanced usage patterns. Existing documentation has been updated to reflect the new API changes, tracking integrations, and dependency updates.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • CHANGELOG.md
    • Added IterationState.learner_id field to IterationState for identifying parallel learners.
    • Changed parallel learner start() methods to return AsyncIterator[IterationState] for real-time state streaming.
    • Extracted internal asyncio.Queue-based fan-in pattern into a shared _stream_parallel helper.
    • Deprecated ParallelActiveLearner.teach(), ParallelReinforcementLearner.learn(), and ParallelUQLearner.teach() in favor of the start() async-iterator API.
    • Added RHAPSODY backend integration, migrating execution backends from radical.asyncflow to rhapsody-py.
    • Introduced .pre-commit-config.yaml with docformatter, ruff, standard file checks, actionlint, GitHub workflow validation, and typos.
    • Integrated pre-commit as a required CI job in tests.yml.
    • Added new tutorials: 03-highly-parallel-surrogates and 04-al-algorithm-selector.
    • Updated core learner logic to be asyncio-based for better concurrency.
    • Updated minimum Python version to 3.10, dropping 3.9 support.
    • Explicitly scoped setuptools to the rose package to prevent accidental inclusion of tutorials/ and examples/.
    • Adjusted Ruff configuration for line length and ML naming conventions.
    • Fixed unquoted $GITHUB_ENV shell variable in GitHub Actions workflows.
  • docs/integrations/clearml.md
    • Added new documentation for ClearML integration, detailing automatic logging of hyperparameters, scalars, and task tags.
  • docs/integrations/mlflow.md
    • Updated MLflow integration documentation to reflect the new MLflowTracker and deprecate manual wiring patterns.
  • docs/user-guide/parallel_learners_docs.md
    • Updated documentation to reflect the new async-iterator API for parallel learners, including learner_id and streaming states.
  • docs/user-guide/tracking.md
    • Added new documentation page detailing the pluggable tracking system, TrackerBase protocol, PipelineManifest, and built-in trackers.
Ignored Files
  • Ignored by pattern: .github/workflows/** (2)
    • .github/workflows/ci.yml
    • .github/workflows/tests.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@AymenFJA AymenFJA requested a review from andre-merzky March 16, 2026 16:10

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a major and beneficial refactoring of the core learner API, transitioning from a blocking teach/learn model to an asynchronous async for state in learner.start() iterator. This change enables real-time monitoring and dynamic control of learning workflows. The parallel learners have been refactored to use this new streaming pattern, improving consistency and code reuse. A new pluggable tracking system has been introduced, cleanly separating observability logic from the core learning loop, with new integrations for MLflow and ClearML. The PR also includes dropping Python 3.9 support, updating dependencies to use rhapsody-py, and adding pre-commit hooks to improve code quality, which has resulted in widespread reformatting. The documentation and examples have been thoroughly updated to reflect these significant improvements. Overall, these are excellent changes that greatly enhance the usability and architecture of the ROSE library.

@AymenFJA AymenFJA marked this pull request as ready for review March 17, 2026 13:37
@AymenFJA AymenFJA merged commit 5adc102 into prototype/raas Mar 25, 2026
10 checks passed
@AymenFJA AymenFJA deleted the fix/raas-main-merge-conflicts branch March 25, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

High Q1 raas ROSE As A Service

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants