Common and NLESolver

A C++ toolkit for nonlinear equation solving, made of three parts:

• NLESolver — interfaces and algorithms for nonlinear equations, nonlinear least squares, scalar equations, array equations and fixed-point problems.
• ModelEnv — a modeling environment under Common/Math/ModelEnv that owns variables, residual constraints, parameters, transforms and scaling, partial masking and Jacobian assembly. It turns a physical model into the vectors and derivatives a solver can consume.
• Common — Eigen linear algebra adapters, Jacobian evaluation, line search, trust region, scaling, logging and utility code shared by NLESolver and ModelEnv.

中文说明 (Chinese)

Highlights

• One interface, many algorithms — create solvers through CreateSolver, CreateScalarSolver and CreateArraySolver; switching algorithm does not require rewriting the problem definition.
• Several Jacobian modes — analytic dense Jacobians, numerical differences and sparse Jacobians, so the cost model can follow the problem size.
• Many solver strategies — Newton, Dogleg, line search, Broyden and trust-region quasi-Newton methods, selectable by stiffness, size and Jacobian availability.
• Production constraints built in — variable bounds, fixed variables, automatic scaling, damping, non-monotone steps, trust-region control and iteration callbacks.
• Diagnosable results — Summary reports residuals, iteration counts, function/Jacobian evaluations, timings and the final exit status.
• Clean component boundary — Common keeps only the math and infrastructure that the solver needs, so it can be used on its own.
• Modeling separated from solving — ModelEnv decides what the model is made of, NLESolver decides how to iterate; changing the model structure does not touch the solving loop.

Features

NLESolver supports these problem forms:

r(x) = 0
min 0.5 * ||r(x)||²
f(x) = 0                       scalar or array problems
x = f(x)                       fixed-point problems

Vector problems can use analytic dense, numerical or sparse Jacobians. The interface also supports variable bounds, fixed variables, automatic scaling, damping, non-monotone steps, iteration callbacks, weights and a solve summary.

The current implementation provides these solvers:

Category	Solvers
Quasi-Newton	Broyden, Thomas, InverseColumn
Trust-region quasi-Newton	TRBroyden, TRCUM
Newton / line search	Newton, Tensor
Dogleg	VanillaDogleg, ExactDogleg, TensorDogleg
Scalar	ITP, ScalarNewton
Fixed point	SuccessiveSubstitution, Wegstein, AAI, QuasiNewton

ExactDogleg suits well-behaved problems with a reliable Jacobian. Without a Jacobian, small problems can use the solver's numerical differences; medium and large problems are usually better served by the quasi-Newton methods.

ModelEnv

ModelEnv lives in Common/Math/ModelEnv and is the modeling layer of this toolkit: it answers "what is the model made of", while NLESolver answers "how do we iterate". Packing and unpacking variables, evaluating constraints, assembling Jacobians, transforms and scaling, converting constraint senses and partial masking are all handled by the environment, so a solver only needs a variable vector, a residual vector and a Jacobian.

Three core entities

Entity	Role	Creation APIs	Notes
Variables	unknowns x	AddVariable, AddScalarVariable, AddVariables	slice, transform, scaling and bounds
Residuals	constraints and objective	AddResidual, AddScalarResidual, AddResiduals, plus the specialized residual APIs	the lhs/rhs relationship
Parameters	parameters	AddParameter, AddScalarParameter, AddParameters	used in evaluation only; not packed with transform or scaling

Every entity is represented by a shared_ptr wrapper returned by the environment, and wrappers are not copyable. The underlying storage can be owned by the wrapper (AddVariable, AddResidual, AddParameter) or by the caller through a pointer or Eigen object (AddVariables, AddResiduals). In the latter case the caller must keep the pointer alive, and after any resize of the external Eigen object the wrapper's ResetCount must be called before re-running FinalizeIndexes.

A constraint reads lhs sense rhs, where sense is Sense::Equal, Sense::GreaterEqual or Sense::LessEqual; residuals with an rhs are converted to a "greater or equal to zero" form as lhs - rhs (rhs - lhs for LessEqual). Transforms and scaling apply per value: first value / scaling_factor, then Transform::Linear, Log, Log1p, SoftOne or Arctan. The environment does not validate the domain of Log, Log1p or SoftOne — use bounds to guarantee it.

Residual types

Creation API	Type	Value evaluation	Jacobian
AddResidual, AddScalarResidual, AddResiduals	General	the caller writes lhs (and rhs if present)	inserted manually via Insert*DerivativeToJacobian
AddLinearResidual	Linear	computed by the environment from the variable mapping and coefficients	generated automatically from the current mapping
AddADResidual	ADResiduals	computed by the environment from a functor plus autodiff	generated automatically
AddScriptResidual	Script	computed by the environment from an exprtk expression	generated automatically
AddComplementarityResidual	Complementarity	computed by the environment from an NCP smoothing function	generated automatically

For everything except General, the environment evaluates values and derivatives; manual insertion of their derivatives is ignored. Complementarity expresses constraints of the form (a + ...) * (b + ...) = 0; the smoothing function and its parameters are set through SetNCPOptions, and the variables involved must have a lower bound of zero. Script uses exprtk expressions, cannot be used as an objective, and must be removed and rebuilt when its mapping changes.

An objective is selected with SetObjective. It must be a scalar Equal General or Linear residual, at most one per environment. Objectives and constraints use separate, zero-based row spaces and are consumed through EvaluateObjective/GetGradient and PackResiduals/GetJacobian respectively.

Typical workflow

1. Declare the required variables, residuals and parameters as class members and keep their wrapper pointers.
2. Add them to the environment with the Add* APIs.
3. Read read-only snapshots with GetInfo; change configuration through controlled APIs such as SetConfiguration, SetEnabled, SetObjective and SetNCPOptions.
4. Optionally mask ranges of variables or residuals with PartiallyDisable, or enable/disable a whole entity with SetEnabled.
5. Call FinalizeIndexes once after the model is built; the environment assigns variable, residual and parameter indices.
6. Get the solver's initial point from PackVariables, and bounds from PackLowerBound / PackUpperBound.
7. Whenever the solver updates x, call UnpackVariables, evaluate the residuals, then PackResiduals or EvaluateObjective to refresh the value caches.
8. Call ReserveNNZ, insert manual derivatives with Insert*DerivativeToJacobian, then consume them with GetJacobian or GetGradient.
9. For debugging, use PrintValue and PrintJacobian; in debug builds GenerateDebugTable produces an entity table.

Contracts and caveats

• Must be held by a shared_ptr — ModelEnv derives from std::enable_shared_from_this and FinalizeIndexes, Merge and the Add* APIs call shared_from_this. Stack allocation is forbidden and copying or moving is disabled.
• Indices can be invalidated — adding, removing, enabling, resizing or changing structural configuration invalidates indices; call FinalizeIndexes again before the next consumption.
• Evaluation order — after the variable values change, constraints must be refreshed with PackResiduals before GetJacobian, and objectives with EvaluateObjective before GetGradient. AD, Complementarity and some Script residuals reuse the caches built by PackResiduals; the framework does not detect skipped evaluations or stale caches.
• Concurrency — neither ModelEnv nor its wrappers are thread-safe. Structural operations (Add, Remove, Merge, FinalizeIndexes, PartiallyDisable, Clear, ...) must run exclusively, and an evaluation cycle from UnpackVariables to GetJacobian must not interleave with another. Only manual derivative production may be parallelized by the caller; concurrent writes to one residual require holding residual->triplet_mutex_, as the insertion APIs do not lock.
• Merge semantics — Merge folds another environment into this one and shares the wrappers; no copies are made and FinalizeIndexes must be called afterwards. The two environments are not independent snapshots that can be interleaved. Fetch is disabled; there is no environment-to- environment copy.
• Size limits — a single wrapper or slice holds at most 10,000 elements, and the total element count of variables, residuals and parameters in one environment is limited to 1,000,000.
• Return value semantics — the evaluation and derivative APIs returning bool only report whether structure, state and process satisfy the contract; they do not guarantee finite numbers. NaN/Inf produced by model values, transforms or floating-point arithmetic propagate to the output.

This repository does not currently ship an adapter that turns a ModelEnv into an NLE::Problem. To integrate one, call UnpackVariables, PackResiduals and GetJacobian from the problem's residual and Jacobian callbacks.

Minimal ModelEnv example

#include <Common/Math/ModelEnv/ModelEnv.hpp>

int main()
{
    // ModelEnv must be held by a shared_ptr, never allocated on the stack
    auto env = std::make_shared<ModelEnv::ModelEnv>();

    // The variable storage is owned by the caller; the environment records the slice,
    // transform, scaling and bounds
    Eigen::VectorXd x(2);
    auto x_var = env->AddVariables(&x, 0, 2, "x",
        ModelEnv::Transform::Linear, 1.0, -10.0, 10.0);

    // Constraint residual: the caller writes the lhs value, the environment applies
    // sense, transform and scaling
    Eigen::VectorXd rx_lhs(2);
    auto rx_var = env->AddResiduals(&rx_lhs, ModelEnv::Sense::Equal, 0, 2, "rx");
    if (VariablesInvalid(x_var) || ResidualsInvalid(rx_var))
        return 1;

    // Finalize once after the model is built, before packing or derivatives
    if (!env->FinalizeIndexes())
        return 1;

    // Initial point for the solver; a value can also be assigned directly
    Eigen::VectorXd x0(2);
    env->PackVariables(x0);
    x0 << 2.0, 2.0;

    // Repeat this evaluation cycle whenever the solver updates x0
    Eigen::VectorXd rx_packed(2);
    env->UnpackVariables(x0);          // write the iterate back into the variables
    rx_lhs(0) = x(0) * x(0) + x(1) - 11.0;
    rx_lhs(1) = x(0) + x(1) * x(1) - 7.0;
    env->PackResiduals(rx_packed);     // the constraint residuals handed to the solver

    // Manual derivatives: reserve space, insert d(lhs)/d(x), then read the Jacobian
    env->ReserveNNZ(4);
    env->InsertDerivativeToJacobian(2.0 * x(0), ModelEnv::Side::Lhs, rx_var, 0, x_var, 0);
    env->InsertDerivativeToJacobian(1.0, ModelEnv::Side::Lhs, rx_var, 0, x_var, 1);
    env->InsertDerivativeToJacobian(1.0, ModelEnv::Side::Lhs, rx_var, 1, x_var, 0);
    env->InsertDerivativeToJacobian(2.0 * x(1), ModelEnv::Side::Lhs, rx_var, 1, x_var, 1);

    Eigen::MatrixXd jacobian(2, 2);
    if (!env->GetJacobian(jacobian))
        return 1;

    return 0;
}

Inserted derivatives are the raw partials without transform or scaling, and they ignore partial masking — the environment applies both.

Getting started

Clone the repository and enter the directory that contains the solution file:

git clone https://github.com/LiuZhexuan/NLESolver.git
cd NLESolver

Build the test program with Visual Studio 2022:

& "$env:ProgramFiles\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\MSBuild.exe" `
    .\NLESolver.sln `
    /t:Build /p:Configuration=test_rel /p:Platform=x64 /m

Run it:

& .\exe\NLESolver_mt_rel_x64.exe

To use NLESolver from your own C++ project, add the repository root and the include directory to the header search path, and link:

lib\NLESolver_mt_rel_x64.lib
lib\Common_mt_rel_x64.lib

For the DLL configuration use dll_rel or dll_dbg; at run time the generated NLESolver DLL must be on a path the program can search.

Repository layout

Common/
  EigenRelated/          linear algebra and matrix containers
  Math/                  Jacobians, line search, Dogleg, trust region, scaling, ...
  Math/ModelEnv/         modeling environment: variables, residuals, parameters, transforms,
                         scaling and Jacobian assembly
  Json/                  Eigen <-> JSON helpers
  Utilities/             bounds checks, logging, export macros and shared utilities

NLESolver/
  Interface/             Problem, Options, Summary and the public factory interfaces
  Solvers/               Newton, Dogleg, quasi-Newton and scalar solvers
  FixedPoint/            fixed-point solvers
  ExampleAndTest/        example problems and the combined test program

include/                 vendored third-party header libraries (see THIRD_PARTY_NOTICES.md)

docs/                    local patch records for vendored libraries

Files under Common/Math/ModelEnv:

File	Contents
ModelEnv.hpp, ModelEnv-inl.hpp	the environment: entity add/remove/find, masking, indices, packing and derivative assembly
Variables.hpp	variable wrapper
Residuals.hpp	residual wrapper and General residuals
Parameters.hpp	parameter wrapper
LinearResiduals.hpp	linear residuals
ADResiduals.hpp	automatic-differentiation residuals
Script.hpp	exprtk expression residuals
Complementarity.hpp	complementarity (NCP) residuals
Transform.hpp, Utilities.hpp	transform/scaling adapters and internal helpers

Requirements and dependencies

The project targets Windows, Visual Studio 2022, MSVC v143, C++17 and x64. The required Eigen, fmt, spdlog, magic_enum, autodiff and exprtk headers are vendored in include/, so no package manager is needed; the Visual Studio C++ toolset, the Windows SDK and OpenMP support are still required. autodiff backs ModelEnv::ADResiduals and exprtk backs ModelEnv::Script.

Vendored library	Upstream	Version / commit	Pinned	License
Eigen	https://gitlab.com/libeigen/eigen	5.0 (5.0.0-dev+master snapshot)	2025-09-21	MPL-2.0
fmt	https://github.com/fmtlib/fmt	10.0.0	2023-05-10	MIT
spdlog	https://github.com/gabime/spdlog	1.12.0	2023-07-11	MIT
nlohmann/json	https://github.com/nlohmann/json	b451735fe7bb3283336f934b3f6cd3f484f73649	2025-06-24	MIT
magic_enum	https://github.com/Neargye/magic_enum	a733a2ea665ca5d72b7270f0334bf2e7b82bd0cc	2025-06-24	MIT
ExprTk	https://github.com/ArashPartow/exprtk	no tag (snapshot)	2023-09-07	MIT
autodiff	https://github.com/autodiff/autodiff	1.1.2	—	MIT

Each library keeps its license text inside its include/<library>/ directory. See THIRD_PARTY_NOTICES.md for the license summary and the upgrade procedure, and docs/LOCAL_PATCHES.md for the patches applied to spdlog, nlohmann/json, magic_enum and autodiff.

The solution configuration selects how the code is built:

• lib_mt_rel / lib_mt_dbg — static library, /MT.
• lib_md_rel / lib_md_dbg — static library, /MD.
• dll_rel / dll_dbg — NLESolver dynamic library, /MD.
• test_rel / test_dbg — NLESolver test program; Common maps to the matching static library configuration.

Other build configurations

To build the static libraries only, change the configuration:

/p:Configuration=lib_mt_rel

The output files are:

lib\Common_mt_rel_x64.lib
lib\NLESolver_mt_rel_x64.lib

For debug builds replace rel with dbg; for the dynamic library use dll_rel or dll_dbg.

Minimal example

#include <NLESolver/Interface/INLESolver.h>
#include <spdlog/sinks/stdout_color_sinks.h>
#include <spdlog/spdlog.h>

int main()
{
    auto logger = spdlog::stdout_color_mt("nle");
    NLE::InitializeSolverLogger(logger);

    auto problem = NLE::CreateFunctorProblem(
        2,
        [](const auto& x, auto rx) -> bool {
            rx(0) = x(0) * x(0) + x(1) - 11.0;
            rx(1) = x(0) + x(1) * x(1) - 7.0;
            return true;
        },
        [](const auto& x, auto jacobian) -> bool {
            jacobian << 2.0 * x(0), 1.0,
                        1.0, 2.0 * x(1);
            return true;
        });

    auto solver = NLE::CreateSolver(NLE::SolverType::ExactDogleg, problem);

    Eigen::VectorXd x(2);
    x << 2.0, 2.0;
    NLE::Options options;
    NLE::Summary summary;
    const auto status = solver->Solve(x, options, summary);

    const bool converged =
        status == NLE::SolverReturnStatus::SolutionReached ||
        status == NLE::SolverReturnStatus::UserSuccess;
    return converged ? 0 : 1;
}

Without an analytic Jacobian, pass nullptr as the third argument and the solver computes a numerical Jacobian according to Options::numerical_jacobian_method_ and the difference step.

For complex problems, derive from NLE::Problem and implement the residual function and GetJacobian. Put problem bounds in lower_bound_ and upper_bound_, and use fixed_variables_indexes_ with fixed_variables_values_ for fixed variables rather than identical bounds.

Results and error handling

Check both the return status and the final residual:

if (status == NLE::SolverReturnStatus::SolutionReached ||
    status == NLE::SolverReturnStatus::UserSuccess) {
    // converged
} else if (!NLE::IsSolverReturnErr(status)) {
    // non-fatal exits such as MaxIterCountReach, StuckInLocalMinimum, CloseToBound
} else {
    // errors such as BoundIllegal, JacobianNotNormal, StepLengthTooSmall
}

InitializeSolverLogger must be called before CreateSolver or Solve. In DLL mode, memory for step information must be pre-allocated as required by the Summary interface, and the NLESolver DLL must be on a path the program can search.

Tests

NLESolver/ExampleAndTest/NLESolverTest.cpp contains the NLE, scalar and fixed-point tests. Run them directly:

& .\exe\NLESolver_mt_rel_x64.exe > .\build\nlesolver-test.log

The process exit code only indicates that the test program ran to completion; it does not mean every algorithm converged from every starting point.

License

This project is licensed under the MIT License — see LICENSE.

Vendored third-party libraries remain under their own licenses; see THIRD_PARTY_NOTICES.md and the license file in each include/ directory.