Xopt Documentation; Small Tweaks

docs/analysis/duplicate_detection.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Duplicate Individual Detection\n",
-    "This notebook demonstrates the use of the method `Population.get_num_unique()` to help detect when populations have duplicate individuals. This method uses `np.round` to deal with floating point accuracy issues."
+    "This page demonstrates the use of the method `Population.get_num_unique()` to help detect when populations have duplicate individuals. This method uses `np.round` to deal with floating point accuracy issues."
    ]
   },
   {

docs/analysis/inverted_generational_distance_analysis.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Analysis of Inverted Generational Distance\n",
-    "This notebook demonstrates how optimization algorithm benchmarking data can be loaded from disk, metrics (such as inverted generational distance) can be calculated for all reported populations, and the resulting data analyzed for comparisons between experiements. We demonstrate the process by comparing three algorithms against eachother on a single test problem. The resulting comparisons are displayed and it is shown how the table can be exported to latex for publication."
+    "This page demonstrates how optimization algorithm benchmarking data can be loaded from disk, metrics (such as inverted generational distance) can be calculated for all reported populations, and the resulting data analyzed for comparisons between experiements. We demonstrate the process by comparing three algorithms against eachother on a single test problem. The resulting comparisons are displayed and it is shown how the table can be exported to latex for publication."
    ]
   },
   {

docs/container_objects.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Container Objects\n",
-    "Some objects are included in ParetoBench to enable users to manipulate and save data related to multi-objective optimizations. We use this notebook to explain some of their usage."
+    "Some objects are included in ParetoBench to enable users to manipulate and save data related to multi-objective optimizations. We use this page to explain some of their usage."
    ]
   },
   {

docs/ext/xopt.ipynb

@@ -0,0 +1,285 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0",
+   "metadata": {},
+   "source": [
+    "# Xopt External Connector\n",
+    "ParetoBench includes code to use its benchmark problems in [Xopt](https://xopt.xopt.org/) and load data output from some of its multiobjective genetic algorithms.\n",
+    "On this page, we will explore the use of the following ParetoBench features.\n",
+    "- `XoptProblemWrapper`: Use ParetoBench problems with Xopt.\n",
+    "- `import_nsga2_history_dir`: Load the output of one or more runs of `NSGA2Generator` into a `paretobench.History`.\n",
+    "- `import_cnsga_history`: Load the output of `CNSGAGenerator` into a `paretobench.History`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import tempfile\n",
+    "import shutil\n",
+    "import numpy as np\n",
+    "import os\n",
+    "from xopt.base import Xopt\n",
+    "from xopt.evaluator import Evaluator\n",
+    "from xopt.generators.ga.cnsga import CNSGAGenerator\n",
+    "from xopt.generators.ga.nsga2 import NSGA2Generator\n",
+    "\n",
+    "from paretobench.ext.xopt import (\n",
+    "    XoptProblemWrapper,\n",
+    "    import_cnsga_history,\n",
+    "    import_nsga2_history_dir,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Make a directory for our optimization data (cleaned up at end of notebook)\n",
+    "dir = tempfile.mkdtemp()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3",
+   "metadata": {},
+   "source": [
+    "### Using ParetoBench Problems in Xopt\n",
+    "In Xopt, problems are described by a VOCS object with the problems variables, objectives, and constraints as well as an evaluation function which maps dicts of variables to dicts of objectives / constraints. `XoptProblemWrapper` adapts any ParetoBench problem to this interface. The VOCS object is available from the wrapper's `vocs` property and the wrapper itself may be passed directly to Xopt's `Evaluator`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Wrap one of ParetoBench's problems for use in Xopt\n",
+    "prob = XoptProblemWrapper.from_line_fmt(\"ZDT3 (n=3)\")\n",
+    "print(prob)\n",
+    "\n",
+    "# The VOCS object is constructed from the problem's variable bounds, objectives, and constraints\n",
+    "prob.vocs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# The wrapper is a callable using Xopt's dict -> dict convention\n",
+    "print(prob({\"x0\": 1.0, \"x1\": 1.5, \"x2\": 2.0}))\n",
+    "\n",
+    "# Arrays are also accepted for use with vectorized evaluators\n",
+    "print(prob({\"x0\": np.array([1.0, 0.5]), \"x1\": np.array([1.5, 0.75]), \"x2\": np.array([2.0, 1.0])}))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6",
+   "metadata": {},
+   "source": [
+    "### Loading Data from NSGA2Generator\n",
+    "`import_nsga2_history_dir` may be used to load optimization data from the `output_dir` of `NSGA2Generator` into a `paretobench.History` object. We will now run a short optimization on the wrapped problem and load the data to demonstrate the use of the function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Generate optimization data for us to load later\n",
+    "population_size = 32\n",
+    "n_generations = 32\n",
+    "output_dir = os.path.join(dir, \"nsga2_zdt3\")\n",
+    "\n",
+    "# Set up and run the optimizer while writing data to `output_dir`\n",
+    "xx = Xopt(\n",
+    "    generator=NSGA2Generator(vocs=prob.vocs, output_dir=output_dir, population_size=population_size),\n",
+    "    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),\n",
+    "    vocs=prob.vocs,\n",
+    ")\n",
+    "for _ in range(n_generations):\n",
+    "    xx.step()\n",
+    "\n",
+    "# To avoid errors when running in notebook\n",
+    "xx.generator.close_log_file()\n",
+    "\n",
+    "print(f\"Ran NSGA2 for {n_generations} generations\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load the run into a History object\n",
+    "hist = import_nsga2_history_dir(output_dir, problem=\"ZDT3 (n=3)\")\n",
+    "\n",
+    "# Show some stats from it\n",
+    "print(hist)\n",
+    "print(f\"Number of reports: {len(hist)}\")\n",
+    "print(f\"Individuals in final report: {len(hist.reports[-1])}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# The method also allows for easy plotting of optimization\n",
+    "# Here we show all individuals along with the true Pareto front\n",
+    "import_nsga2_history_dir(output_dir, problem=\"ZDT3 (n=3)\").plot_obj_scatter(show_pf=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "10",
+   "metadata": {},
+   "source": [
+    "Multiple runs of `NSGA2Generator` created by restarting from checkpoints can be combined into a single `History` object. Pass a list of the output directories (in the order they were run) or a glob pattern such as `\"my_optimization_*\"` to `import_nsga2_history_dir`. The VOCS in each directory must match and the generation numbering must be continuous across the runs (as produced by checkpoint restarts)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "11",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Select the latest checkpoint file from the run\n",
+    "checkpoint_dir = os.path.join(output_dir, \"checkpoints\")\n",
+    "checkpoint_file = max(\n",
+    "    (os.path.join(checkpoint_dir, f) for f in os.listdir(checkpoint_dir)),\n",
+    "    key=os.path.getmtime,\n",
+    ")\n",
+    "\n",
+    "# Run for more generations after checkpoint\n",
+    "xx = Xopt(\n",
+    "    generator=NSGA2Generator(vocs=prob.vocs, checkpoint_file=checkpoint_file),\n",
+    "    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),\n",
+    "    vocs=prob.vocs,\n",
+    ")\n",
+    "for _ in range(n_generations):\n",
+    "    xx.step()\n",
+    "\n",
+    "# To avoid errors when running in notebook\n",
+    "xx.generator.close_log_file()\n",
+    "\n",
+    "# Show the newly created file\n",
+    "print(f\"Completed {n_generations} more generations\")\n",
+    "print(\"Contents of project directory:\")\n",
+    "for fname in sorted(os.listdir(dir)):\n",
+    "    print(\"  - \" + fname)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "12",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Plot data from both runs (Note \"*\" character after output directory to match all outputs)\n",
+    "import_nsga2_history_dir(os.path.join(dir, \"nsga2_zdt3*\"), problem=\"ZDT3 (n=3)\").plot_obj_scatter(show_pf=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "13",
+   "metadata": {},
+   "source": [
+    "### Loading Data from CNSGAGenerator\n",
+    "`CNSGAGenerator` saves each generation to a timestamped CSV file (`cnsga_population_*.csv`) when run with the `output_path` option. The function `import_cnsga_history` loads all of the population files in a directory into a `History` object. The population files do not contain the problem definition, so the VOCS object must be passed in as well. A path to a JSON file of the VOCS or the run's YAML config file (through the `config` keyword) also works."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "14",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Generate optimization data using CNSGA\n",
+    "output_path = os.path.join(dir, \"cnsga\")\n",
+    "xx = Xopt(\n",
+    "    generator=CNSGAGenerator(vocs=prob.vocs, output_path=output_dir, population_size=population_size),\n",
+    "    evaluator=Evaluator(function=prob, max_workers=population_size, vectorized=True),\n",
+    "    vocs=prob.vocs,\n",
+    ")\n",
+    "for _ in range(n_generations):\n",
+    "    xx.step()\n",
+    "\n",
+    "print(f\"Ran CNSGA for {n_generations} generations\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "15",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load the run into a History object\n",
+    "import_cnsga_history(output_dir, vocs=prob.vocs, problem=\"ZDT3 (n=3)\").plot_obj_scatter(show_pf=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "16",
+   "metadata": {},
+   "source": [
+    "### Cleanup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "17",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Clean up optimization data\n",
+    "shutil.rmtree(dir)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "paretobench",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/index.md

@@ -53,7 +53,7 @@ Tools for plotting the data from multi-objective optimization algorithms are als
     - Color coding or animation for showing multiple populations
     - Markers and alpha to distinguish non-dominated / infeasible solutions
 
-See more information in the following notebooks.
+See more information on the following pages.
 
 - [Plotting populations](plotting/plotting_populations.ipynb)
 - [Plotting histories](plotting/plotting_histories.ipynb)

docs/plotting/plotting_histories.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Plotting of History Objects\n",
-    "In Paretobench, `History` objects contain multiple populations in an ordered list of \"reports\" which represent the progress of an optimization on solving a problem. This notebook demonstrates some options for plotting this data."
+    "In Paretobench, `History` objects contain multiple populations in an ordered list of \"reports\" which represent the progress of an optimization on solving a problem. This page demonstrates some options for plotting this data."
    ]
   },
   {

docs/plotting/plotting_metrics.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "# Plotting Metrics Across Histories\n",
-    "This notebooks demonstrates the use of `plot_metric_history` which allows for the plotting the evolution of metrics over each population within a `History` object."
+    "This page demonstrates the use of `plot_metric_history` which allows for the plotting the evolution of metrics over each population within a `History` object."
    ]
   },
   {

docs/plotting/plotting_populations.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Plotting of Population Objects\n",
-    "This notebook demonstrates some of the functions available in ParetoBench for plotting the data in `Population` objects."
+    "This page demonstrates some of the functions available in ParetoBench for plotting the data in `Population` objects."
    ]
   },
   {

docs/problems/problems.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "# Benchmark Problems\n",
-    "This notebook demonstrates the usage of the ParetoBench `Problem` objects which implement common multi-objective benchmark problems in a convient vectorized format."
+    "This page demonstrates the usage of the ParetoBench `Problem` objects which implement common multi-objective benchmark problems in a convient vectorized format."
    ]
   },
   {
@@ -134,7 +134,7 @@
    "metadata": {},
    "source": [
     "### Analytic Pareto Fronts\n",
-    "Some of the test problems come with analytic Pareto fronts. These are useful for the evaluating convergence of optimization algorithms under test. In this notebook, we demonstrate how to query points from the Pareto fronts. Problems which can return points from the Pareto front inherit from `ProblemWithPF` and problems with a fixed number of points on the Pareto front inherit from `ProblemWithFixedPF`."
+    "Some of the test problems come with analytic Pareto fronts. These are useful for the evaluating convergence of optimization algorithms under test. On this page, we demonstrate how to query points from the Pareto fronts. Problems which can return points from the Pareto front inherit from `ProblemWithPF` and problems with a fixed number of points on the Pareto front inherit from `ProblemWithFixedPF`."
    ]
   },
   {

mkdocs.yml

@@ -34,3 +34,5 @@ nav:
   - Analysis:
       - Duplicate Detection: analysis/duplicate_detection.ipynb
       - IGD Analysis: analysis/inverted_generational_distance_analysis.ipynb
+  - External Connectors:
+      - Xopt: ext/xopt.ipynb

pyproject.toml

@@ -38,6 +38,7 @@ docs = [
   "mkdocs",
   "mkdocs-jupyter",
   "mkdocs-material",
+  "xopt"
 ]
 
 [tool.ruff]