Part of #23740.
SPMDEngine/DaskEngine/RayEngine.__init__ need num_py_executors,
hardware_binding, memory_resource_config, quent_context, kvikio_nthreads, and
allow_gpu_sharing before ConfigOptions/StreamingExecutor can exist, to actually
build a thread pool or bind a GPU. Build one StreamingOptions at construction, read
these off it, and write the fully-resolved dict back as executor_options/
engine_options before calling super().__init__, so self.config is a frozen, correct
snapshot from the moment the engine exists instead of a partially-resolved one
ConfigOptions.from_polars_engine has to patch up on every query. Once every field
resolves to a concrete value this way, _reset() can just merge the caller's overrides
onto the previous config instead of needing per-field carry-forward logic.
Part of #23740.
SPMDEngine/DaskEngine/RayEngine.__init__neednum_py_executors,hardware_binding,memory_resource_config,quent_context,kvikio_nthreads, andallow_gpu_sharingbeforeConfigOptions/StreamingExecutorcan exist, to actuallybuild a thread pool or bind a GPU. Build one
StreamingOptionsat construction, readthese off it, and write the fully-resolved dict back as
executor_options/engine_optionsbefore callingsuper().__init__, soself.configis a frozen, correctsnapshot from the moment the engine exists instead of a partially-resolved one
ConfigOptions.from_polars_enginehas to patch up on every query. Once every fieldresolves to a concrete value this way,
_reset()can just merge the caller's overridesonto the previous config instead of needing per-field carry-forward logic.