Multi-objective Pareto-frontier examples (workforce + portfolio)#151
Multi-objective Pareto-frontier examples (workforce + portfolio)#151cafzal wants to merge 16 commits into
Conversation
…io QP duals) Signed-off-by: cafzal <cameron.afzal@gmail.com>
…-sum gotcha demo) Signed-off-by: cafzal <cameron.afzal@gmail.com>
…older READMEs Signed-off-by: cafzal <cameron.afzal@gmail.com>
… sweeps Signed-off-by: cafzal <cameron.afzal@gmail.com>
Signed-off-by: cafzal <cameron.afzal@gmail.com>
…onvention) Signed-off-by: cafzal <cameron.afzal@gmail.com>
Adds `cuopt-multi-objective-exploration` — a concept skill for problems with **two or more objectives and no fixed weighting**, where the user wants to see the tradeoff instead of accepting one weighted answer. It turns repeated single-objective cuOpt solves into a Pareto frontier (payoff table → ε-constraint / weighted-sum sweep → filter dominated) and supplies the discipline to read it: exchange rates, knee points, deferring the final choice. It adds no solver features and invents no API; it sits above the api-* and formulation skills, orchestrating the solves they already cover. ### cuOpt-specific correctness points - ε-constraining is most natural on **linear** objectives, so a quadratic objective (risk `xᵀΣx`) can simply stay the objective. A convex one can also be ε-constrained directly: cuOpt routes `xᵀQx ≤ ε` through the barrier solver as a second-order cone. - PDLP warm-start is LP-only; MILP frontier points are optimal **to the gap you set**, since each solve gets a time limit. - A hard constraint (coverage floor, budget, fairness cap) is often a latent objective — promote it to a swept ε-constraint (when its level was an assumption, not a firm limit). ### User testing Used the skill to enrich two existing examples and confirmed it drove the right calls — companion PR NVIDIA/cuopt-examples#151: - **Workforce (MILP)** — promoted two hard constraints into tradeoffs (coverage → cost vs. coverage; the `max_shifts` cap → cost vs. fairness), chose ε-constraint over weighted-sum, anchored objective ranges, filtered dominated points, capped each solve, and reported no MILP duals. - **Portfolio (QP)** — recognized the base notebook's hand-coded target-return loop as ε-constraint and added what it omits: the return-constraint dual (shadow price d(variance)/d(return)), with the PDLP-tolerance caveat. The skill drives the *method*; runnable code still needs `cuopt-numerical-optimization-api-python` plus the worked notebooks, which are Colab-GPU validated (T4, clean end-to-end). ### Validation & gating Registered in `AGENTS.md` + `marketplace.json`; `ci/utils/validate_skills.sh` passes. The contribution is `SKILL.md` + `evals/evals.json` — `BENCHMARK.md`, the skill card, and `skill.oms.sig` are generated by the NVSkills onboarding pipeline. The official NVSkills-Eval is the gate (PENDING). Authors: - Cameron Afzal (https://github.com/cafzal) Approvers: - Miles Lubin (https://github.com/mlubin) - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #1355
Signed-off-by: cafzal <cameron.afzal@gmail.com>
rgsl888prabhu
left a comment
There was a problem hiding this comment.
Few minor suggestions, but test looks good. Awesome work @cafzal.
| "cell_type": "markdown", | ||
| "id": "f330297d", | ||
| "metadata": {}, | ||
| "source": "# Portfolio Optimization \u2014 the Frontier via the Skill, + Shadow Prices (cuOpt QP)\n\nThe base `QP_portfolio_optimization` notebook **hand-codes** an efficient-frontier sweep (a manual loop over target returns). This sibling shows that following the `cuopt-multi-objective-exploration` skill **recreates that frontier as a named, systematic workflow** \u2014 anchor each objective \u2192 \u03b5-constraint sweep (the return floor is the parametric bound) \u2192 filter dominated \u2192 read the frontier \u2014 with less ad-hoc scaffolding, and **adds the one thing the manual sweep omits**: the return-constraint **dual** (shadow price d(variance)/d(return)).\n\nSo the two examples are complementary tests of the skill: here it **reproduces** an existing frontier (return vs risk) with less manual work and surfaces the duals; the workforce MILP (`workforce_optimization/workforce_optimization_multiobjective.ipynb`) is the **net-new** case \u2014 and the deliberate contrast is that **a QP has constraint duals, an integer program does not.**" |
There was a problem hiding this comment.
I may be missing something but the notebook doesn't really show how the skill was used, it just shows the output. What is the take home for readers of the notebook?
There was a problem hiding this comment.
Reworked both intros around the method as explicit steps (recognize → constrain one → sweep → read the dual) and added a "Takeaway" section; cut the "follows the skill" framing. The key point is now explicit: a hand-coded target-return loop already is an ε-constraint sweep.
| "cell_type": "markdown", | ||
| "id": "13784d94", | ||
| "metadata": {}, | ||
| "source": "# Workforce Optimization \u2014 Multi-Objective with cuOpt\n\nThe base `workforce_optimization_milp` notebook minimizes labor cost with coverage **hard-constrained** \u2014 it returns **one plan**. But that plan answers only *\"cheapest way to fully staff.\"* A planner usually faces a **tradeoff with no fixed weighting**: *how much coverage is worth how much cost?* and *how much does fairness cost?* A single solve hides that; you get one point on a curve you can't see.\n\nThis notebook follows the `cuopt-multi-objective-exploration` skill to turn the single solve into the **whole tradeoff curve**, so the planner can see the options and choose. Two tradeoffs, both built by promoting one of the base model's hard constraints into an objective:\n\n1. **cost vs. coverage** \u2014 relax `coverage == required` and sweep a coverage floor.\n2. **cost vs. fairness** \u2014 sweep the base model's fixed `max_shifts` cap." |
There was a problem hiding this comment.
What does it mean for the notebook to follow the skill? The notebook doesn't show how to use the skill; it just shows the output.
There was a problem hiding this comment.
Same resolution. Please let me know if the reframe better illustrates the workflow.
…t, drop cross-references, fix QP-dual wording Signed-off-by: cafzal <cameron.afzal@gmail.com>
…VIDIA#151) Signed-off-by: cafzal <cameron.afzal@gmail.com>
Signed-off-by: cafzal <cameron.afzal@gmail.com>
…k-through Signed-off-by: cafzal <cameron.afzal@gmail.com>
Signed-off-by: cafzal <cameron.afzal@gmail.com>
Signed-off-by: cafzal <cameron.afzal@gmail.com>
…h base style Signed-off-by: cafzal <cameron.afzal@gmail.com>
| "cell_type": "markdown", | ||
| "id": "f330297d", | ||
| "metadata": {}, | ||
| "source": "# Multi-Objective Portfolio Optimization with cuOpt Python API\n\nThis notebook demonstrates how to use the cuOpt Python API to trace the **efficient frontier** of a mean-variance portfolio \u2014 the multi-objective core of portfolio optimization, where there's no single best balance of return and risk, only a curve of optimal tradeoffs.\n\nThe base `QP_portfolio_optimization` notebook solves for individual portfolios; here we sweep the whole frontier with the **\u03b5-constraint method** and read each point's **sensitivity** (the return-constraint dual):\n\n1. **Two objectives that conflict** \u2014 maximize return, minimize variance.\n2. **Keep one as the objective, constrain the other** \u2014 minimize variance subject to `return \u2265 \u03b5`.\n3. **Sweep \u03b5** across the achievable return range; each solve is one frontier point.\n4. **Read the frontier** \u2014 and, because this is a continuous QP, read each point's **dual**: the sensitivity d(variance)/d(return), how much extra variance one more unit of return costs.\n\nA useful thing to notice: the base notebook already loops over target returns, so it is **already doing this** \u2014 naming the method is what lets you add the dual and reuse the recipe on any two-objective problem. (This workflow is also packaged as the `cuopt-multi-objective-exploration` skill.)" |
There was a problem hiding this comment.
| "source": "# Multi-Objective Portfolio Optimization with cuOpt Python API\n\nThis notebook demonstrates how to use the cuOpt Python API to trace the **efficient frontier** of a mean-variance portfolio \u2014 the multi-objective core of portfolio optimization, where there's no single best balance of return and risk, only a curve of optimal tradeoffs.\n\nThe base `QP_portfolio_optimization` notebook solves for individual portfolios; here we sweep the whole frontier with the **\u03b5-constraint method** and read each point's **sensitivity** (the return-constraint dual):\n\n1. **Two objectives that conflict** \u2014 maximize return, minimize variance.\n2. **Keep one as the objective, constrain the other** \u2014 minimize variance subject to `return \u2265 \u03b5`.\n3. **Sweep \u03b5** across the achievable return range; each solve is one frontier point.\n4. **Read the frontier** \u2014 and, because this is a continuous QP, read each point's **dual**: the sensitivity d(variance)/d(return), how much extra variance one more unit of return costs.\n\nA useful thing to notice: the base notebook already loops over target returns, so it is **already doing this** \u2014 naming the method is what lets you add the dual and reuse the recipe on any two-objective problem. (This workflow is also packaged as the `cuopt-multi-objective-exploration` skill.)" | |
| "source": "# Multi-Objective Portfolio Optimization with cuOpt Python API\n\nThis notebook demonstrates how to use the cuOpt Python API to trace the **efficient frontier** of a mean-variance portfolio \u2014 the multi-objective core of portfolio optimization, where there's no single best balance of return and risk, only a curve of optimal tradeoffs.\n\nThe base `QP_portfolio_optimization` notebook solves for individual portfolios; here we sweep the whole frontier with the **\u03b5-constraint method** and read each point's **sensitivity** (the return-constraint dual):\n\n1. **Two objectives** \u2014 maximize return, minimize variance.\n2. **Keep one as the objective, constrain the other** \u2014 minimize variance subject to `return \u2265 \u03b5`.\n3. **Sweep \u03b5** across the achievable return range; each solve is one frontier point.\n4. **Read the frontier** \u2014 and, because this is a continuous QP, read each point's **dual**: the sensitivity d(variance)/d(return), how much extra variance one more unit of return costs.\n\nA useful thing to notice: the base notebook already loops over target returns, so it is **already doing this** \u2014 naming the method is what lets you add the dual and reuse the recipe on any two-objective problem. (This workflow is also packaged as the `cuopt-multi-objective-exploration` skill.)" |
There was a problem hiding this comment.
"that conflict" doesn't add much, the problem simply has two objectives
There was a problem hiding this comment.
Applied. Step 1 is now "Two objectives: maximize return, minimize variance."
| "cell_type": "markdown", | ||
| "id": "6888f83f", | ||
| "metadata": {}, | ||
| "source": "## Step 4 \u2014 read the frontier\n\n- The **frontier** (left) is the return-vs-risk Pareto set \u2014 every point is a min-variance portfolio for its return floor. There's no single \"best\"; you choose where on the curve to sit.\n- The **dual** (right) is the **sensitivity** d(variance)/d(return): how much extra variance each additional unit of return costs. It rises along the frontier \u2014 the marginal cost of return steepens, which is exactly where a knee analysis pays off.\n\n### Takeaway \u2014 reusing this on your own problem\nTwo competing objectives and a solver for one of them is all you need: keep one objective, turn the other into a swept constraint (`f\u2082 \u2265 \u03b5` or `\u2264 \u03b5`), solve across the range, and read the frontier. If your code already loops over a target value, that loop **is** an \u03b5-constraint sweep \u2014 name it, collect the non-dominated points, and (for an LP or QP) read the constraint's dual for the marginal exchange rate.\n\n### Notes\n- **Synthetic data** \u2014 the base notebook's simulated universe; demonstrates the method.\n- **Barrier solver** \u2014 a quadratic objective is solved by cuOpt's barrier (interior-point) method (1e-8 relative accuracy by default), so the dual is accurate to that tolerance, not exact arithmetic; points reported `PrimalFeasible` rather than `Optimal` are flagged above and could be tightened or dropped.\n- **Duals need continuity** \u2014 these sensitivities exist because the portfolio is a continuous QP; an integer program would expose none." |
There was a problem hiding this comment.
No mention is needed for the numerical precision of the duals from barrier. This is standard.
There was a problem hiding this comment.
Dropped, from both the Notes and the Step 3 text.
| "cell_type": "markdown", | ||
| "id": "6888f83f", | ||
| "metadata": {}, | ||
| "source": "## Step 4 \u2014 read the frontier\n\n- The **frontier** (left) is the return-vs-risk Pareto set \u2014 every point is a min-variance portfolio for its return floor. There's no single \"best\"; you choose where on the curve to sit.\n- The **dual** (right) is the **sensitivity** d(variance)/d(return): how much extra variance each additional unit of return costs. It rises along the frontier \u2014 the marginal cost of return steepens, which is exactly where a knee analysis pays off.\n\n### Takeaway \u2014 reusing this on your own problem\nTwo competing objectives and a solver for one of them is all you need: keep one objective, turn the other into a swept constraint (`f\u2082 \u2265 \u03b5` or `\u2264 \u03b5`), solve across the range, and read the frontier. If your code already loops over a target value, that loop **is** an \u03b5-constraint sweep \u2014 name it, collect the non-dominated points, and (for an LP or QP) read the constraint's dual for the marginal exchange rate.\n\n### Notes\n- **Synthetic data** \u2014 the base notebook's simulated universe; demonstrates the method.\n- **Barrier solver** \u2014 a quadratic objective is solved by cuOpt's barrier (interior-point) method (1e-8 relative accuracy by default), so the dual is accurate to that tolerance, not exact arithmetic; points reported `PrimalFeasible` rather than `Optimal` are flagged above and could be tightened or dropped.\n- **Duals need continuity** \u2014 these sensitivities exist because the portfolio is a continuous QP; an integer program would expose none." |
There was a problem hiding this comment.
"Duals need continuity" this is LP 101, I don't think we need to mention this here.
| "cell_type": "markdown", | ||
| "id": "29d59f8d", | ||
| "metadata": {}, | ||
| "source": "## Notes & takeaway\n\n**Takeaway \u2014 reusing this on your own problem.** When a single-objective model has a hard constraint whose level was *assumed* \u2014 a coverage target, a per-resource cap, a budget \u2014 that constraint is a hidden objective. Promote it to a swept \u03b5-constraint, collect the non-dominated points, and read the exchange rate off the frontier. The same recipe traces any such tradeoff.\n\n- **Synthetic data** \u2014 the base notebook's toy roster; this demonstrates the *method*, not a staffing study.\n- **Optimal to the gap** \u2014 each point is solved under a `time_limit`; every solve here returned `Optimal` (0 `FeasibleFound`), so each is optimal to cuOpt's MIP gap (exact here, since labor cost is integer-valued).\n- **No duals for a MILP** \u2014 an integer program has no constraint duals, so the marginal cost of coverage is read off the frontier itself (above)." |
There was a problem hiding this comment.
I don't think any of these three bullet points are necessary, they're all obvious/standard.
There was a problem hiding this comment.
Dropped all three. Kept the "Takeaway" section.
Signed-off-by: cafzal <cameron.afzal@gmail.com>
Multi-objective (Pareto frontier) examples — companion to the
cuopt-multi-objective-explorationskillTwo examples that extend existing folders to demonstrate multi-objective Pareto-frontier exploration with cuOpt — the workflow added as the
cuopt-multi-objective-explorationskill in NVIDIA/cuopt#1355 (discussion NVIDIA/cuopt#1351).workforce_optimization/workforce_optimization_multiobjective.ipynb(MILP) — turns the cost-minimizing workforce model into a tradeoff surface: cost vs. coverage (relaxcoverage == requiredand sweep a coverage floor) and cost vs. fairness (promote the base model's fixedmax_shiftscap to a swept ε-constraint — a fixed constraint treated as a candidate objective). Reads the frontier as an exchange rate (marginal $ per shift), caps every MILP solve with atime_limit, and shows that an integer program has no constraint duals. ε-constraint is the default; weighted-sum is a one-line method note, not a demo.portfolio_optimization/QP_portfolio_frontier_duals.ipynb(QP) — recognizes the base QP notebook's hand-coded target-return loop as the ε-constraint method, rebuilds it as the named workflow, and adds the piece the manual sweep omits: the return-constraint dual (shadow price d(variance)/d(return)) along the efficient frontier, with the PDLP-tolerance caveat.Both reuse the base notebooks' data, run on cuOpt alone (Colab GPU), and follow the repo's notebook idiom (GPU check →
cuopt-cu12install → solve). Notebooks ship output-stripped (repo convention — every existing cuopt-examples notebook has 0 cell outputs); the run evidence is below.User testing
Both notebooks were built by following the skill, then run end-to-end on Colab T4, cuOpt 26.4.0 — clean, no API errors:
Optimal(0FeasibleFound). A single solve was only ever the right-most point.max_shiftscap (the constraint-as-objective move): full coverage holds at $468 down to a cap of 11, then $470 / $473 / $484 at caps 10 / 9 / 8, and goes infeasible at ≤ 7 — a clean price-of-fairness curve plus a feasibility cliff.Optimal(0PrimalFeasible), so the PDLP-tolerance caveat is mild here.Notes
Optimal(0FeasibleFound) — optimal to cuOpt's gap tolerance (exact here, since labor cost is integer-valued); the per-solvetime_limitis a guard that didn't bind.PrimalFeasiblepoint is flagged (none here).Companion to the now-merged
cuopt-multi-objective-explorationskill (NVIDIA/cuopt#1355).