[Core] Add initial foundational configurations and flags for fork server by chiayi · Pull Request #67356 · ray-project/ray

chiayi · 2026-10-23T22:54:32Z

Description

Sets up configuration and fork-safety needed for the Python worker fork server.

No behavior change by default (RAY_enable_worker_fork_server=false).

Key Changes

Add preload_modules option to RuntimeEnv and fork_server_socket_path to RuntimeEnvContext
Add enable_worker_fork_server and worker_fork_server_timeout_ms config flags and forward fork server env vars to RuntimeEnvAgent
Extract initialize_default_worker() in default_worker.py so forked workers can initialize directly
Fix post-fork worker state (re-read internal config flags, reseed C++ PRNG on PID change, sync torch thread count, and only save worker PGID when pgid == pid)

Related issues

#67446 - Python worker fork server

Additional information

N/A

gemini-code-assist · 2026-10-23T22:54:36Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

gemini-code-assist · 2026-10-23T22:56:15Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

cursor

Stale Bugbot comment from a previous run.

ryanaoleary · 2026-10-26T21:02:38Z

    os.environ["OMP_NUM_THREADS"] = str(omp_num_threads)
+    # If torch was pre-imported before fork(), its OpenMP thread pool is already
+    # initialized and ignores subsequent os.environ["OMP_NUM_THREADS"] updates.
+    if "torch" in sys.modules:


Prefer a constant str list (or just str if we can validate torch is the only sys module that needs this branching logic) here rather than hard-coding the "torch" module name

Added a THREAD_POOL_MODULES var to iterator over.

ryanaoleary · 2026-10-26T21:30:54Z

created a Ray issue here: #67446 to link in the PRs so that they're easier to track

cursor

Stale Bugbot comment from a previous run.

ryanaoleary · 2026-10-27T22:27:17Z

cc: @MengjinYan @edoakes for Ray core review, or if you know someone else able to approve that it'd be good to work with on this

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Reviewed by Cursor Bugbot for commit b97eb39. Configure here.}

Kunchd

Thanks for the PR! The first milestone looks good, and I look forward to the future milestones! I mainly left some questions to clarify my understanding of the change.

Kunchd · 2026-11-03T23:24:21Z

+            )
+        if stripped not in validated:
+            validated.append(stripped)
+    return validated


If the validated module doesn't actually exist, what's our plan for surfacing the import error to the user?

This is actually being taken care of in the follow-up PR (#67514 but WIP), it will catch and log preload import errors in the fork server logs without failing server startup. If the task or actor imports that missing module at runtime, Python will raise ModuleNotFoundError to the user. Please let me know if you have thoughts on this!

Kunchd · 2026-11-03T23:52:16Z

    num_threads = str(max(math.floor(num_assigned_cpus), 1))
    for name in unset_env_vars:
        os.environ[name] = num_threads
+    if "OMP_NUM_THREADS" in unset_env_vars:


In the design doc, if I'm understanding the diagram correctly, the forkserver with the specified modules is started before the env vars is configured. If the user specifies OMP_NUM_THREADS to configure torch threads in the runtime env vars, would that configuration be ignored here?

No, it won't be ignored. This current change is only here to handle the case where the user doesn't set OMP_NUM_THREADS and Ray defaults it from num_cpus.

Followup changes will apply env_vars right after fork() and call torch.set_num_threads(...) when OMP_NUM_THREADS is set.

Kunchd · 2026-11-04T00:24:31Z

+        RAY_LOG(WARNING) << "Worker " << pid << " pgid (" << pgid
+                         << ") != pid; skipping SetSavedProcessGroupId to avoid "
+                         << "signaling parent process group.";


Will this branch always trigger for the fork server? If so, this is intended behavior, should we be emitting at INFO level instead?

So this branch won't trigger usually, this is just a safeguard in case setpgid fails unexpectedly. The plan is to place each forked child into its own process group via setpgid so forked workers will also have pgid == pid in normal operation.

Kunchd

Might need to merge master again. But everything looks good!

Signed-off-by: Aaron Liang <aaronliang@google.com>

chiayi requested a review from ryanaoleary October 23, 2026 22:54

chiayi force-pushed the fork-server branch 2 times, most recently from 213b493 to 9d6cc18 Compare October 23, 2026 22:56

chiayi marked this pull request as ready for review October 23, 2026 22:56

chiayi requested a review from a team as a code owner October 23, 2026 22:56

cursor Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread src/ray/util/random.h Resolved

ray-gardener Bot added core Issues that should be addressed in Ray Core community-contribution Contributed by the community labels Oct 24, 2026

chiayi force-pushed the fork-server branch 2 times, most recently from ff3dc87 to 0512a5e Compare October 26, 2026 19:26

Yicheng-Lu-llll assigned ryanaoleary Oct 26, 2026

ryanaoleary reviewed Oct 26, 2026

View reviewed changes

Comment thread python/ray/_raylet.pyx Resolved

ryanaoleary reviewed Oct 26, 2026

View reviewed changes

Comment thread src/ray/util/random.h Resolved

ryanaoleary reviewed Oct 26, 2026

View reviewed changes

Comment thread src/ray/util/random.h Resolved

ryanaoleary reviewed Oct 26, 2026

View reviewed changes

Comment thread src/ray/common/ray_config.cc Resolved

ryanaoleary mentioned this pull request Oct 26, 2026

[Core] Python worker fork server #67446

Open

1 of 5 tasks

chiayi force-pushed the fork-server branch 2 times, most recently from 59b838c to 5bcd33a Compare October 26, 2026 23:56

chiayi requested a review from ryanaoleary October 26, 2026 23:59

ryanaoleary reviewed Oct 27, 2026

View reviewed changes

Comment thread src/ray/raylet/worker_pool.cc Resolved

ryanaoleary approved these changes Oct 27, 2026

View reviewed changes

chiayi force-pushed the fork-server branch from 5bcd33a to 1eb777a Compare October 27, 2026 20:40

cursor Bot reviewed Oct 27, 2026

View reviewed changes

Comment thread python/ray/_raylet.pyx Resolved

chiayi force-pushed the fork-server branch 2 times, most recently from 6dfd31c to be33589 Compare October 27, 2026 22:26

chiayi force-pushed the fork-server branch from 9a0c8f2 to b97eb39 Compare November 3, 2026 19:59

cursor Bot reviewed Nov 3, 2026

View reviewed changes

Comment thread src/ray/util/random.h Resolved

chiayi force-pushed the fork-server branch from b97eb39 to af43d24 Compare November 3, 2026 22:59

Kunchd added the go add ONLY when ready to merge, run all tests label Nov 3, 2026

Kunchd reviewed Nov 4, 2026

View reviewed changes

chiayi force-pushed the fork-server branch from af43d24 to a117330 Compare November 4, 2026 19:21

Kunchd approved these changes Nov 5, 2026

View reviewed changes

chiayi force-pushed the fork-server branch 12 times, most recently from 46e4591 to 63d36fa Compare November 8, 2026 02:37

ryanaoleary mentioned this pull request Nov 9, 2026

[Prototype] Add Python worker fork server #67272

Draft

4 of 7 tasks

chiayi force-pushed the fork-server branch 7 times, most recently from 6efb430 to 1f098a5 Compare November 10, 2026 18:34

Add initial foundations for fork server

5f0fe46

Signed-off-by: Aaron Liang <aaronliang@google.com>

chiayi force-pushed the fork-server branch from 1f098a5 to 5f0fe46 Compare November 10, 2026 19:39

MengjinYan merged commit 11a0a60 into ray-project:master Nov 11, 2026
6 checks passed

Conversation

chiayi commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key Changes

Related issues

Additional information

Uh oh!

gemini-code-assist Bot commented Oct 23, 2026

Uh oh!

gemini-code-assist Bot commented Oct 23, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ryanaoleary Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

chiayi Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanaoleary commented Oct 26, 2026

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ryanaoleary commented Oct 27, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Kunchd left a comment

Choose a reason for hiding this comment

Uh oh!

Kunchd Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

chiayi Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kunchd Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

chiayi Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Kunchd Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

chiayi Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Kunchd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

chiayi commented Oct 23, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

chiayi Nov 4, 2026 •

edited

Loading