Nunn is a free and open-source machine learning library written in C++20, distributed under the MIT License.
The library aims to be compact, readable, and practical — a codebase you can actually study while experimenting with neural networks and other machine learning algorithms in modern C++.
- Features
- Building and testing
- Feedforward networks
- Recurrent networks
- Associative memory
- Unsupervised / generative
- Convolutional networks
- Transformer
- Reinforcement learning
- Scripts
- Demos and tools
- Documentation
- Perceptron — single neuron, MSE or cross-entropy loss, momentum
- MlpNN — classic fully connected MLP with per-layer activations
- MlpMatrixNN — Eigen 3.4 backed MLP with mini-batch SGD/Adam and optional ArrayFire/OpenCL acceleration
- VanillaRnn — Elman-style RNN with truncated BPTT
- GRU — Gated Recurrent Unit with truncated BPTT
- LSTM — Long Short-Term Memory with truncated BPTT
- Hopfield — energy-based associative memory
- Autoencoder — symmetric encoder–decoder built on MlpMatrixNN
- Rbf — Radial Basis Function network (Gaussian centres + SGD output weights)
- Conv1DLayer / MaxPool1DLayer / ConvNet — 1D convolutional pipeline with im2col and end-to-end backprop
- LayerNorm / SelfAttentionLayer / TransformerBlock / MiniTransformer — decoder-only transformer with multi-head causal attention and autoregressive generation
- DQN — Deep Q-Network with experience replay buffer and frozen target network
- Q-learning and SARSA tabular reinforcement learning
- 342 GoogleTest unit tests; all network classes are fully tested
- Cross-platform: Windows, Linux, macOS
Requires CMake 3.14+ and a C++20 compiler. Eigen 3.4 and nlohmann/json are fetched automatically via FetchContent; GoogleTest is fetched when tests are enabled.
cmake -S . -B build
cmake --build build --config Release
ctest --test-dir build -C ReleasePass -DNUNN_BUILD_TESTS=OFF to skip the test suite. OpenCL support is attempted
by default through ArrayFire (-DNUNN_ENABLE_OPENCL=ON); if ArrayFire/OpenCL is
not installed, MlpMatrixNN::ComputeBackend::Auto falls back to Eigen/CPU. Use
-DNUNN_ENABLE_OPENCL=OFF to build a CPU-only library.
On Windows, build-opencl.ps1 configures the project with OpenCL enabled and
defaults to the standard ArrayFire install directory:
.\build-opencl.ps1
.\build-opencl.ps1 -Target ocr_test
.\build-opencl.ps1 -ArrayFireRoot "C:\Program Files\ArrayFire\v3" -CleanCacheWhen ArrayFire/OpenCL is found, the build deploys the required runtime DLLs beside
OpenCL-enabled executables so ocr_test, mnist_test, and nunn_tests can run
without manually editing PATH.
Packages install the core runtime demos and tools selected by the top-level CMake install rules under <install-root>\bin. The source-tree build provides the complete examples gallery.
On Windows, the Start menu contains shortcuts for the graphical demos, the test
runner, documentation, and Nunn Developer Command Prompt, which opens a console
with the nunn bin directory already on PATH. On Linux/macOS, the package
installs nunn-dev-shell.sh and nunn-run-tests.sh in bin for the same
workflow. From that shell, examples such as mnist_test, xor_test,
nunn_tests, and net2json can be launched directly.
The GitHub Wiki contains the extended documentation layer for nuNN:
The wiki complements this README with book-derived explanations, diagrams, training diagnostics, MNIST/OCR notes, and links from each concept to the corresponding source files and examples.
The project is accompanied by the book Fundamentals of Machine Learning: Algorithms and Applications in C++, available in English and Italian editions:
- English Kindle: https://www.amazon.com/dp/B0GY9L7N22
- English paperback: https://www.amazon.com/dp/B0H7KQCFJY
- Italian Kindle: https://www.amazon.it/dp/B0H6Q12LVJ
- Italian paperback: https://www.amazon.it/dp/B0DF69MPZF
The simplest trainable unit: a single neuron with a weighted sum, a bias, and a nonlinearity. It can only learn linearly separable functions (e.g. AND, OR) but serves as the foundation for understanding gradient descent.
Learning rule (online SGD):
ŷ = σ(w · x + b)
δ = (t − ŷ) · σ'(ŷ) MSE loss
δ = t − ŷ cross-entropy + sigmoid (derivative cancels)
w += lr · δ · x + momentum · Δw_prev
b += lr · δ
#include "nu_perceptron.h"
nu::StepFunction step(0.5, 0.0, 1.0);
nu::Perceptron p(
2, // number of inputs
0.1, // learning rate
step,
nu::CostFunction::MSE,
0.9 // momentum
);
p.setInputVector({1.0, 0.0});
p.backPropagate(1.0); // train toward target 1
p.setInputVector({1.0, 0.0});
p.feedForward(); // inference
double label = p.getSharpOutput();Demo: and_test — learns the AND function in a handful of epochs.
A fully connected multilayer network trained with online SGD and backpropagation. Topology is expressed as a vector<size_t> where the first element is the input size, the last is the output size, and everything in between defines hidden layers.
Supported per-layer activations: Sigmoid, Tanh, ReLU, LeakyReLU, Linear.
Cost functions: MSE, CrossEntropy (CE requires Sigmoid on the output layer).
#include "nu_mlpnn.h"
nu::MlpNN nn(
{784, 300, 10}, // input → hidden → output
0.05, // learning rate
0.9, // momentum
nu::CostFunction::CrossEntropy
);
nn.setInputVector(sample);
nn.feedForward();
nn.backPropagate(target);MlpTrainer wraps the epoch loop with an early-stopping criterion:
nu::MlpTrainer trainer(nn, /*max_epochs*/ 30, /*min_err*/ 0.01);
trainer.runTraining(dataset, costCallback);Model states can be saved and reloaded through streams:
#include <fstream>
{
std::ofstream out("model.json");
nn.toJson(out);
}
nu::MlpNN loaded;
{
std::ifstream in("model.json");
loaded.loadJson(in);
}Demo: xor_test — the classic non-linearly separable problem.
Demo: mnist_test — MNIST digit recognition (784→300→10, ~98% accuracy).
XOR cannot be solved by a linear model, but a two-layer MLP with a small hidden layer can learn the required non-linear boundary:
x1 | x2 | y
---+----+---
0 | 0 | 0
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0
nu::MlpNN nn({2, 2, 1}, 0.4, 0.9);
nu::MlpTrainer trainer(nn, 20000, 0.01);
trainer.runTraining<TrainingSet>(
{{{0,0},{0}}, {{0,1},{1}}, {{1,0},{1}}, {{1,1},{0}}},
[](nu::MlpNN& net, const auto& target) {
return net.calcMSE(target);
}
);A drop-in, higher-performance alternative to MlpNN. Weights are stored as Eigen::MatrixXd tensors, enabling vectorised GEMV (single sample) and GEMM (mini-batch) operations via Eigen 3.4.
The public interface mirrors MlpNN; the key addition is trainBatch():
#include "nu_mlpmatrixnn.h"
using LC = nu::MlpMatrixNN::LayerConfig;
nu::MlpMatrixNN nn(
{LC(784), LC(300, nu::Activation::ReLU),
LC(10, nu::Activation::Sigmoid)},
0.05, 0.9,
nu::CostFunction::CrossEntropy
);
// Mini-batch SGD uses parallel input and target containers.
std::vector<std::vector<double>> inputs;
std::vector<std::vector<double>> targets;
nn.trainBatch(inputs, targets);mnist_test exposes CPU/GPU backend selection via flags:
mnist_test # MlpMatrixNN, Auto GPU/CPU, batch 100
mnist_test --mlp # classic MlpNN, online SGD
mnist_test --backend cpu # MlpMatrixNN, force Eigen/CPU
mnist_test --backend opencl # MlpMatrixNN, require ArrayFire/OpenCL
mnist_test --batch 1 # MlpMatrixNN, Auto GPU/CPU, online SGDMlpMatrixNN JSON persistence is tested across training, save, reload, and
recognition. OpenCL-backed networks synchronise device weights before JSON
serialisation and after JSON load, so a model trained on GPU behaves the same
after reloading from disk.
Both recurrent classes (VanillaRnn, Lstm) share the same public interface and are interchangeable. They use truncated BPTT (backpropagation through time) with per-element gradient clipping, and support two output modes:
RnnOutput |
Loss | Typical use |
|---|---|---|
Linear |
MSE | regression, sequence prediction |
Softmax |
Cross-entropy | classification, language modelling |
The simplest recurrent architecture. At each time step the hidden state is updated from the current input and the previous hidden state, then passed through an output projection:
h_t = tanh(Wx · x_t + Wh · h_{t-1} + b_h)
y_t = f_out(Wy · h_t + b_y)
Wx maps the input, Wh feeds the hidden state back into itself.
The hidden state carries information forward in time; training "unrolls" the computation graph for truncate steps at a time (truncated BPTT).
Strengths: simple, fast, good for short dependencies.
Weakness: gradients vanish quickly over long sequences (the LSTM addresses this).
#include "nu_rnn.h"
nu::VanillaRnn rnn(
/*inputSize*/ 1,
/*hiddenSize*/ 32,
/*outputSize*/ 1,
/*lr*/ 0.005,
/*gradClip*/ 5.0,
/*outMode*/ nu::RnnOutput::Linear
);
// Training: feed a full sequence, get mean loss and update weights
rnn.resetState();
double loss = rnn.bptt(inputs, targets, /*truncate*/ 25);
// Inference: one step at a time
rnn.step({0.42});
double y = rnn.getOutput()[0];Weight initialisation: Xavier normal for Wx, Wh, Wy; biases zero.
Gradient clipping is per-element: each gradient component is clamped to [-gradClip, +gradClip].
Introduced by Cho et al. (2014), the GRU simplifies the LSTM by merging the input and forget gates into a single update gate and removing the separate cell state. The result is a model with ~25% fewer parameters that often matches LSTM performance on short-to-medium sequences.
r_t = σ(Wr·x_t + Ur·h_{t-1} + b_r) reset gate
z_t = σ(Wz·x_t + Uz·h_{t-1} + b_z) update gate
g_t = tanh(Wh·x_t + Uh·(r_t ⊙ h_{t-1}) + b_h) candidate hidden state
h_t = (1 − z_t) ⊙ h_{t-1} + z_t ⊙ g_t new hidden state
y_t = f_out(Wy·h_t + b_y)
Reset gate r_t controls how much of the previous hidden state leaks into the candidate: when r_t ≈ 0 the candidate ignores history and can write a fresh value; when r_t ≈ 1 the candidate behaves like a VanillaRnn step.
Update gate z_t interpolates between the old hidden state and the candidate: z_t ≈ 0 keeps the previous state unchanged (long-range memory); z_t ≈ 1 replaces it entirely (fast adaptation).
Implementation detail: input weights for all three gates are stacked as W [3·nh × ni]. The recurrent weights for r and z share a single GEMV via Urz [2·nh × nh]; the candidate recurrent weight Uh [nh × nh] is applied separately to r_t ⊙ h_{t-1}.
#include "nu_gru.h"
nu::Gru gru(
/*inputSize*/ 2,
/*hiddenSize*/ 32,
/*outputSize*/ 1,
/*lr*/ 0.005,
/*gradClip*/ 5.0,
/*outMode*/ nu::RnnOutput::Linear
);
gru.resetState();
double loss = gru.bptt(inputs, targets, /*truncate*/ 25);
gru.step({0.7, 1.0});
double y = gru.getOutput()[0];The Gru API is identical to VanillaRnn and Lstm.
The Long Short-Term Memory adds a cell state c_t — a separate memory line that flows through time with only element-wise operations (no matrix multiply). Three sigmoid gates control what information enters, leaves, and is forgotten from the cell:
i_t = σ(Wi·x_t + Ui·h_{t-1} + b_i) input gate — what to write
f_t = σ(Wf·x_t + Uf·h_{t-1} + b_f) forget gate — what to erase (b_f init = 1)
o_t = σ(Wo·x_t + Uo·h_{t-1} + b_o) output gate — what to expose
g_t = tanh(Wg·x_t + Ug·h_{t-1} + b_g) cell candidate
c_t = f_t ⊙ c_{t-1} + i_t ⊙ g_t update cell state
h_t = o_t ⊙ tanh(c_t) hidden state exposed to output
y_t = f_out(Wy · h_t + b_y)
The forget gate bias is initialised to 1 so the network starts by remembering everything, which helps gradients flow at the beginning of training.
Implementation detail: the four gate weight matrices are stacked vertically as W [4·nh × ni] and U [4·nh × nh], allowing a single GEMV per step (pre = W·x + U·h + b) followed by a split into four blocks. This reduces kernel-launch overhead and keeps the hot path cache-friendly.
#include "nu_lstm.h"
nu::Lstm lstm(
/*inputSize*/ 28, // vocabulary size (one-hot)
/*hiddenSize*/ 64,
/*outputSize*/ 28,
/*lr*/ 0.005,
/*gradClip*/ 5.0,
/*outMode*/ nu::RnnOutput::Softmax
);
lstm.resetState();
double loss = lstm.bptt(inputs, targets, /*truncate*/ 25);
lstm.step(oneHot('a'));
char next = sampleFromSoftmax(lstm.getOutput());The Lstm API is identical to VanillaRnn — resetState(), step(), bptt(), reshuffleWeights(), getOutput(), getHidden() — so the two classes can be used interchangeably (e.g. via a template or the --lstm flag in the demos).
Trains a recurrent network to predict the next sample of a sine wave given the current one. After training it runs autoregressively: each predicted value is fed back as the next input, testing whether the learned dynamics are self-sustaining.
rnn_sine # VanillaRnn, 1500 epochs, hidden=32, lr=0.005
rnn_sine --lstm # LSTM, same defaults
rnn_sine --lstm 2000 64 0.003 # --lstm [epochs] [hidden] [lr]The adding problem (Hochreiter & Schmidhuber, 1997) is a standard benchmark for comparing recurrent architectures on long-range memory.
Each input sequence has length T. Every element is a pair (value, marker) where value ∈ [0, 1] and marker ∈ {0, 1}. Exactly two positions are marked — one in the first half, one in the second half. The network must output the running cumulative sum of marked values at each step, reaching the total sum at the final step.
This requires selective memory: values at unmarked positions must be ignored; values at marked positions must be remembered and accumulated, even when far apart in the sequence.
All three architectures (VanillaRnn, GRU, LSTM) are trained on the same dataset and compared side-by-side:
rnn_adding # seq_len=20, hidden=32, epochs=500, lr=0.005
rnn_adding 30 64 800 0.003 # seq_len hidden epochs lrExample output (300 epochs, seq_len=20, hidden=32):
Baseline MAE (predict 0.5) = 0.5447
VanillaRnn epoch 299 train loss 0.00507 test MAE 0.143
GRU epoch 299 train loss 0.00088 test MAE 0.054
LSTM epoch 299 train loss 0.00150 test MAE 0.055
GRU and LSTM both significantly outperform VanillaRnn on this task, as expected for a problem that requires retaining specific values over many steps.
Trains a recurrent network to predict the next character given the current one, accumulating context in the hidden (and cell) state. After training it generates text autoregressively, sampling from the softmax output with a temperature parameter.
The corpus is an embedded English pangram repeated several times (~28-character vocabulary); enough to observe the network learning basic character-level statistics within a few hundred epochs.
rnn_char # VanillaRnn, 800 epochs, hidden=64
rnn_char --lstm # LSTM
rnn_char --lstm 1200 128 80 0.6 # --lstm [epochs] [hidden] [gen_len] [temperature]Lower temperature → more repetitive but coherent text.
Higher temperature → more varied but noisier output.
A Hopfield network is an energy-based recurrent model that stores patterns as stable attractors of a dynamical system. Given a noisy or incomplete input, it converges to the nearest stored pattern by repeatedly applying an update rule until the energy stops decreasing.
The network has N fully connected neurons with symmetric weights (no self-connections). Patterns are stored using the Hebb rule:
W_ij = (1/N) Σ_p ξ_i^p · ξ_j^p (i ≠ j)
Retrieval: start from a corrupted input and update neurons asynchronously until convergence.
Storage capacity is approximately 0.138 · N patterns before retrieval becomes unreliable.
Demo: hopfield_test — stores and recalls a set of 100-pixel binary images.
A symmetric encoder–decoder built on top of MlpMatrixNN. The encoder compresses the input to a low-dimensional latent code; the decoder reconstructs the input from that code. Training minimises MSE reconstruction loss end-to-end.
#include "nu_autoencoder.h"
nu::Autoencoder ae(
16, // input and reconstruction size
{8, 4}, // encoder; 4 is the bottleneck
nu::Activation::Tanh,
0.005
);
double loss = ae.train(dataset, 500);
auto code = ae.encode(dataset.front());
auto recon = ae.decode(code);Demo: ae_demo — trains on sinusoid fragments; prints latent codes and reconstruction error.
A Radial Basis Function network. Hidden units compute Gaussian similarity to a fixed set of centres:
h_j(x) = exp( -‖x - c_j‖² / (2 σ_j²) )
Centres are placed by random sampling (fitCenters); the output layer is then trained with SGD. Sigma is set by the heuristic d_max / sqrt(2 · n_centers).
#include "nu_rbf.h"
nu::Rbf rbf(
/*inputSize*/ 1,
/*numCenters*/ 12,
/*outputSize*/ 1,
/*lr*/ 0.01,
/*outMode*/ nu::RnnOutput::Linear
);
rbf.fitCenters(data); // place centres by random sampling
rbf.train(inputs, targets, 500); // SGD on output weights
auto y = rbf.forward({0.5}); // inferenceDemo: rbf_demo — fits sin(x) over [0, 2π] with 12 RBF centres; prints train/test MSE.
A 1D convolutional pipeline with Eigen-backed im2col forward and col2im backward passes, and a builder API for stacking layers before a fully-connected head.
Conv1DLayer — valid-padding stride-1 convolution:
outLen = inLen - kernelSize + 1
Y = W · Xcol + b (W [outCh × inCh·K], Xcol [inCh·K × outLen])
Initialisation: He for ReLU/LeakyReLU, Xavier otherwise.
MaxPool1DLayer — non-overlapping max pooling; saves argmax mask for gradient routing in backward.
ConvNet — builder that chains conv/pool layers and attaches an MlpMatrixNN head:
#include "nu_convnet.h"
using LC = nu::MlpMatrixNN::LayerConfig;
nu::ConvNet cnn(1, 16); // 1 channel, 16 time steps
cnn.addConv1D(8, 5, nu::Activation::Tanh, 0.005);
cnn.addMaxPool1D(4); // flatFeatureSize = 8*3 = 24
cnn.setFCHead({
LC(cnn.flatFeatureSize()),
LC(16, nu::Activation::Tanh),
LC(2, nu::Activation::Sigmoid)
}, 0.005);
double loss = cnn.train(x, target);
auto out = cnn.predict(x);End-to-end backprop flows from the FC head via getInputGradient() back through the conv/pool stack.
Demo: cnn_seq — classifies 1D sine signals by frequency (1 vs 2 cycles over 16 samples, Gaussian noise σ=0.15); reaches >80% test accuracy in 500 epochs.
A decoder-only transformer for character-level (or token-level) language modelling. Architecture (Pre-LN style):
Token IDs → Embedding + Sinusoidal PositionalEncoding
→ N × TransformerBlock( LN → MH-Attn(causal) → residual
→ LN → FFN(ReLU) → residual )
→ Output projection → logits [seqLen × vocabSize]
LayerNorm normalises each row of the token matrix; gamma/beta are learned with analytically-correct backward.
SelfAttentionLayer uses h per-head projections W_Q^h, W_K^h, W_V^h ∈ R^{d×dk} where dk = d/h. The causal mask sets scores above the diagonal to −∞ before softmax. Full backward through the softmax Jacobian, Q·K^T, and all projections.
#include "nu_transformer.h"
nu::MiniTransformer model(
/*vocabSize*/ 23, // unique chars
/*seqLen*/ 32, // context window
/*dModel*/ 64,
/*numHeads*/ 4, // dk = 16
/*dFF*/ 128,
/*numLayers*/ 2,
/*lr*/ 0.005
);
double loss = model.train(inputs, targets); // cross-entropy loss
auto logits = model.forward(tokens); // [seqLen × vocabSize]
std::mt19937 rng(42);
auto gen = model.generate(prompt, 80, /*temperature*/ 0.8, &rng);Demo: transformer_char — trains on a ~300-character Shakespeare excerpt; cross-entropy drops from ~3.1 to ~0.10 in 1000 epochs, generates recognisable continuations.
Nunn includes tabular implementations of two fundamental RL algorithms.
An off-policy temporal-difference method. The agent learns an action-value function Q(s, a) — the expected cumulative reward for taking action a in state s — by bootstrapping from the Bellman equation:
Q(s, a) ← Q(s, a) + α · [r + γ · max_a' Q(s', a') − Q(s, a)]
The policy is ε-greedy: with probability ε the agent explores randomly; otherwise it picks argmax_a Q(s, a). Q-learning converges to the optimal policy regardless of which policy is used to collect data (off-policy).
An on-policy TD method. The update uses the action actually taken next (a'), not the greedy best:
Q(s, a) ← Q(s, a) + α · [r + γ · Q(s', a') − Q(s, a)]
SARSA is more conservative than Q-learning in stochastic environments because it accounts for the exploration policy during learning.
Demos:
- Maze — navigate from start to goal on a grid
- Path finder — find shortest paths under obstacles
Replaces the tabular Q-table with a neural network (MlpMatrixNN), enabling RL in continuous or high-dimensional state spaces. Key components:
ExperienceReplayBuffer<State, Action>— fixed-capacity ring buffer; uniform random sampling of mini-batches breaks temporal correlations.Dqn— maintains two networks: a main network updated every step and a target network whose weights are frozen and synced everytargetUpdateFreqlearn steps. The Bellman target is:
q[a] = r + γ · max_{a'} Q_target(s', a')
Gradient is computed only for the taken action; all other outputs keep target = Q_main(s) so their gradients are zero.
#include "nu_dqn.h"
using LC = nu::MlpMatrixNN::LayerConfig;
nu::Dqn agent(
{ LC(2), LC(32, nu::Activation::ReLU),
LC(32, nu::Activation::ReLU), LC(4, nu::Activation::Linear) },
/*lr*/ 0.001, /*bufferCapacity*/ 10000,
/*batchSize*/ 32, /*gamma*/ 0.99, /*targetUpdateFreq*/ 100
);
int action = agent.selectAction(state, epsilon);
double loss = agent.learn(state, action, reward, nextState, done);Demo: dqn_maze — 5×5 grid world; state = normalised (row, col); 4 directional actions; solves >90% of episodes after training.
Ready-made scripts for running and comparing all RNN architectures are provided in scripts/rnn/.
scripts/rnn/powershell/
_common.ps1 — shared helpers (exe discovery, Run-Example function)
run_sine.ps1 — sine-wave prediction: train and compare models
run_char.ps1 — character-level language model: train and generate text
run_adding.ps1 — adding problem benchmark (all models, single run)
run_all.ps1 — run all three examples in sequence
Common options accepted by most scripts:
| Flag | Description |
|---|---|
-Quick |
Reduce epoch count for a fast smoke-test |
-Model vanilla|gru|lstm|all |
Select one or all architectures (default: all) |
-Epochs N |
Override epoch count |
-Hidden N |
Hidden units (default varies per script) |
-Lr F |
Learning rate |
Examples:
# Quick sanity check — all three models, 400 epochs
.\scripts\rnn\powershell\run_sine.ps1 -Quick
# Full sine comparison with custom params
.\scripts\rnn\powershell\run_sine.ps1 -Epochs 2000 -Hidden 64 -Lr 0.003
# Train GRU char model only
.\scripts\rnn\powershell\run_char.ps1 -Model gru -Epochs 1200 -Hidden 128
# Adding problem benchmark (longer sequences)
.\scripts\rnn\powershell\run_adding.ps1 -SeqLen 30 -Hidden 64 -Epochs 800
# Run everything (quick mode)
.\scripts\rnn\powershell\run_all.ps1 -QuickMirror of the PowerShell scripts with -- style flags:
scripts/rnn/bash/
_common.sh — shared helpers
run_sine.sh — sine-wave prediction
run_char.sh — character-level language model
run_adding.sh — adding problem benchmark
run_all.sh — run all examples
# Quick sanity check
bash scripts/rnn/bash/run_sine.sh --quick
# Full comparison, custom params
bash scripts/rnn/bash/run_sine.sh --epochs 2000 --hidden 64 --lr 0.003
# Single model
bash scripts/rnn/bash/run_char.sh --model gru --epochs 1200
# Adding benchmark with longer sequences
bash scripts/rnn/bash/run_adding.sh --seq-len 30 --hidden 64 --epochs 800
# Run everything
bash scripts/rnn/bash/run_all.sh --quickTraining scripts for the MNIST examples are in scripts/mnist/powershell/ and scripts/mnist/bash/.
run_all trains all activation/cost-function combinations and prints a comparison table with BER and throughput.
.\scripts\mnist\powershell\run_all.ps1 -Quickbash scripts/mnist/bash/run_all.sh --quick| Demo | Model | Description |
|---|---|---|
and_test |
Perceptron | AND function (linearly separable) |
xor_test |
MlpNN | XOR function (non-linearly separable) |
mnist_test |
MlpNN / MlpMatrixNN | MNIST digit recognition (784→300→10) |
ocr_test |
MlpNN / MlpMatrixNN | Interactive handwritten digit recognition and MNIST training |
rnn_sine |
VanillaRnn / GRU / LSTM | Sine-wave next-step prediction |
rnn_char |
VanillaRnn / GRU / LSTM | Character-level language model |
rnn_adding |
VanillaRnn / GRU / LSTM | Adding problem benchmark (selective memory) |
ae_demo |
Autoencoder | Sinusoid fragment compression and reconstruction |
rbf_demo |
Rbf | Sin(x) regression with Gaussian RBF centres |
cnn_seq |
ConvNet | 1D frequency classification (1 vs 2 cycles) |
dqn_maze |
Dqn | 5×5 grid world with DQN and experience replay |
transformer_char |
MiniTransformer | Char-level LM on Shakespeare excerpt |
tictactoe |
MlpNN | Tic Tac Toe via neural network |
winttt |
MlpNN | Interactive Windows Tic Tac Toe |
hopfield_test |
Hopfield | Pattern recall from noisy input |
maze |
Q-learning / SARSA | Grid-world navigation |
path_finder |
Q-learning / SARSA | Shortest-path under obstacles |
nunn_topo |
topology tool | Visualise MLP topology from JSON, legacy .net, or explicit layer sizes |
The MNIST dataset contains 60,000 training and 10,000 test images of handwritten digits (28×28 grayscale, flattened to 784 inputs). The mnist_test tool defaults to MlpMatrixNN with backend auto and batch size 100:
mnist_test -p /path/to/mnist # MlpMatrixNN, Auto GPU/CPU, batch 100
mnist_test -p /path/to/mnist --mlp # classic MlpNN, online SGD
mnist_test -p /path/to/mnist --backend cpu # force Eigen/CPU
mnist_test -p /path/to/mnist --backend opencl # require ArrayFire/OpenCL
mnist_test -p /path/to/mnist --batch 1 # MlpMatrixNN, Auto GPU/CPU, online SGDMore information: http://yann.lecun.com/exdb/mnist/
ocr_test loads legacy .net models and JSON models, performs real-time
handwritten digit recognition, and can train a new MNIST model directly from the
Train menu. The MNIST training dialog supports MlpNN and MlpMatrixNN, backend
selection (Auto, CPU, OpenCL), persisted MNIST/save paths, mini-batch
training, and a cost-convergence chart with labelled axes. Models trained from
the dialog are saved as JSON and can be reloaded for recognition.
nunn_topo turns an MLP topology into a Graphviz diagram. It can read modern
JSON models, legacy .net models, or an explicit topology passed on the command
line:
nunn_topo --load examples/ocr_test/nn_125hl.json --save ocr-topology.svg
nunn_topo --load examples/tictactoe/tictactoe.net --save tictactoe.dot
nunn_topo --topology 784,300,10 --save mnist.pngDOT output works without extra tools:
nunn_topo --topology 2,3,1
nunn_topo --topology 2,3,1 --save xor.dotSVG, PNG, and PDF output require Graphviz dot in PATH:
nunn_topo --load model.json --format svg --save model.svgLarge networks are compacted by default so MNIST-sized topologies remain
readable. Use --full to draw every node and every connection, or
--max-nodes N to tune the compact view.