Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,58 @@ Run the following commands in the project root:

If you add a new feature, make sure to include appropriate tests in `src/**/tests.rs` or as integration tests.

## Troubleshooting the build

### `ffi-backend` / `--all-features` fails in `build.rs` with a bindgen panic

The default build is **pure Rust and needs no C++ toolchain** — you only hit
this when you opt into the C++ FreakMatcher via `--features ffi-backend`
(or `--all-features`, or `dual-mode`, which implies it).

**Symptom**

```text
error: failed to run custom build command for `webarkitlib-rs`
thread 'main' panicked at .../bindgen-0.72.1/lib.rs:917:13:
assertion `left == right` failed: "x86_64-pc-windows-msvc" "x86_64-pc-windows-msvc"
left: 4
right: 8
```

**Cause** — `LIBCLANG_PATH` points at a `libclang` built for a **32-bit**
target, so bindgen sees a pointer width of `4` while the host target
(`x86_64`) expects `8`. The confusing part is that both sides of the
assertion print the same triple; only the widths differ.

A common way to end up here on Windows is having a cross-compilation
toolchain's `libclang` on `LIBCLANG_PATH` — for example the Xtensa clang
that ships with the [esp-rs](https://github.com/esp-rs) toolchain
(`~/.rustup/toolchains/esp/xtensa-esp32-elf-clang/...`), which targets the
32-bit ESP32.

**Fix** — point `LIBCLANG_PATH` at an `x86_64` `libclang` for the shell you
build this repo in:

```powershell
$env:LIBCLANG_PATH = "C:\Program Files\LLVM\bin"
cargo clippy --workspace --all-targets --all-features -- -D warnings
```

The `x64` libclang bundled with Visual Studio works too:
`C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\x64\bin`.

**Set it per-session, not globally**, if you also do embedded work — the
cross-toolchain needs its own `LIBCLANG_PATH`. For the same reason we do
**not** ship a `.cargo/config.toml` `[env]` override: it would need
`force = true` to beat an already-set variable, and cargo's `[env]` has no
per-OS conditionals, so it would push a Windows path onto Linux CI and
override its `libclang-dev`.

> Worth fixing rather than ignoring: without it you cannot run the
> `--all-features` gate locally, so breakage that only exists behind
> `ffi-backend` (e.g. lints firing on bindgen-generated code) reaches CI
> instead of your machine.

## Commit Message Conventions

To maintain a clean and automated release history, this project strictly adheres to the [Conventional Commits](https://www.conventionalcommits.org/) specification.
Expand Down
3 changes: 3 additions & 0 deletions crates/core/src/kpm/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,11 @@ pub struct FeaturePoint {
/// Z coordinate is always `0.0` for planar targets.
#[derive(Debug, Default, Clone, PartialEq)]
pub struct Point3d {
/// X coordinate in millimetres.
pub x: f32,
/// Y coordinate in millimetres.
pub y: f32,
/// Z coordinate in millimetres. Always `0.0` for planar targets.
pub z: f32,
}

Expand Down
10 changes: 9 additions & 1 deletion crates/core/src/kpm/freak/gaussian_pyramid.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,21 @@ pub enum GaussianPyramidError {
EmptyImage,
/// Input image is smaller than 5x5; the binomial filter needs 2 border
/// pixels on each side.
ImageTooSmall { rows: usize, cols: usize },
ImageTooSmall {
/// Row count of the offending input image.
rows: usize,
/// Column count of the offending input image.
cols: usize,
},
/// `num_octaves` was 0 — a pyramid must have at least one octave.
ZeroOctaves,
/// Halving for `octave` would produce a level smaller than 5x5.
OctaveTooSmall {
/// Index of the octave that could not be built.
octave: usize,
/// Row count the octave would have had.
rows: usize,
/// Column count the octave would have had.
cols: usize,
},
}
Expand Down
13 changes: 11 additions & 2 deletions crates/core/src/kpm/freak/hough.rs
Original file line number Diff line number Diff line change
Expand Up @@ -306,11 +306,13 @@ impl BinParams {
/// Accumulates votes for similarity transformations in discretized 4D space.
pub struct HoughSimilarityVoting {
params: BinParams,
/// Object center in reference image.
/// Object center in the reference image (x, in pixels).
pub center_x: f32,
/// Object center in the reference image (y, in pixels).
pub center_y: f32,
/// Reference image dimensions.
/// Reference image width in pixels.
pub ref_image_width: i32,
/// Reference image height in pixels.
pub ref_image_height: i32,
/// Vote map: bin_index → vote count.
///
Expand Down Expand Up @@ -537,9 +539,13 @@ impl HoughSimilarityVoting {
/// C equivalent: vision::FeaturePoint
#[derive(Clone, Debug, Copy)]
pub struct FeaturePoint {
/// Horizontal position in pixels.
pub x: f32,
/// Vertical position in pixels.
pub y: f32,
/// Dominant orientation in radians.
pub angle: f32,
/// Scale (pyramid octave level) at which the keypoint was detected.
pub scale: f32,
/// True if this is a maxima, false if a minima (used to filter matches).
pub maxima: bool,
Expand Down Expand Up @@ -570,8 +576,11 @@ pub struct Match {
/// A scored match used internally by Hough voting (carries distance for ranking).
#[derive(Clone, Copy, Debug)]
pub struct HoughMatch {
/// Index into the query image's feature-point list.
pub query_idx: u32,
/// Index into the reference database's feature-point list.
pub ref_idx: u32,
/// Descriptor distance for this match; lower is a closer match.
pub distance: f32,
}

Expand Down
5 changes: 4 additions & 1 deletion crates/core/src/kpm/freak/pyramid.rs
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,10 @@ pub enum PyramidError {
/// Downsampling at this level would produce a 0-sized image.
/// Returned when the input image is too small for the requested
/// number of levels.
LevelTooSmall { level: usize },
LevelTooSmall {
/// Index of the level that could not be built.
level: usize,
},
}

impl std::fmt::Display for PyramidError {
Expand Down
6 changes: 6 additions & 0 deletions crates/core/src/kpm/kpm_ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@
#[cfg(feature = "ffi-backend")]
mod bindings {
#![allow(non_upper_case_globals, non_camel_case_types, non_snake_case)]
// rationale: the contents are generated verbatim by bindgen in build.rs
// from kpm_c_api.h, so there is no source to attach rustdoc to. The
// module-level docs above describe the surface instead; the safe wrapper
// (CppFreakMatcher) is what consumers should use. Needed because the kpm
// tree opts into `#![warn(missing_docs)]` (#226).
#![allow(missing_docs)]
include!(concat!(env!("OUT_DIR"), "/kpm_bindings.rs"));
}

Expand Down
53 changes: 41 additions & 12 deletions crates/core/src/kpm/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,38 +38,67 @@
//! ported from the C++ FreakMatcher library.
//!
//! This module provides a Rust interface to the FREAK-descriptor-based
//! feature matching pipeline used for planar image tracking. It supports
//! two operation modes:
//! feature matching pipeline used for planar image tracking. The matcher is
//! pluggable via the [`FreakMatcherBackend`] trait, with two implementations:
//!
//! - **`ffi-backend`** (default) — delegates to the compiled C++ FreakMatcher
//! static library through a thin `extern "C"` wrapper (`kpm_c_api.h`).
//! - *(future)* **pure-Rust** — a native Rust re-implementation of the same
//! pipeline, swappable via the [`FreakMatcherBackend`] trait.
//! - **pure Rust** ([`RustFreakMatcher`]) — the **default** since M9-3
//! (#142). `cargo build` needs no C++ toolchain (no clang / libclang / cc).
//! - **`ffi-backend`** (`CppFreakMatcher`) — **opt-in**, delegates to the
//! compiled C++ FreakMatcher static library through a thin `extern "C"`
//! wrapper (`kpm_c_api.h`). Enable with `--features ffi-backend`. Used for
//! cross-validation (`dual-mode`, `cross_stack_parity`), not required at
//! runtime.
//!
//! ## Quick start
//!
//! Detect a marker in a grayscale frame with the pure-Rust backend:
//!
//! ```rust,ignore
//! use webarkitlib_rs::kpm::{CppFreakMatcher, KpmHandle};
//! use webarkitlib_rs::kpm::{KpmHandle, RustFreakMatcher};
//! use webarkitlib_rs::kpm::types::KpmRefDataSet;
//!
//! // Create a C++ backend for a 640x480 camera frame.
//! let backend = CppFreakMatcher::new(640, 480).unwrap();
//! // Pure-Rust backend for a 640x480 camera frame.
//! let backend = RustFreakMatcher::new(640, 480)?;
//!
//! // Wrap it in a KpmHandle (the central coordinator).
//! let mut handle = KpmHandle::new(640, 480, None, Box::new(backend));
//!
//! // Load reference data generated by `nft_marker_gen` (.fset3).
//! let ref_data = KpmRefDataSet::load("marker.fset3")?;
//! handle.set_ref_data_set(ref_data)?;
//!
//! // Run detection on one frame; `luma` is width * height grayscale bytes.
//! handle.kpm_matching(&luma)?;
//! if let Some((pose, page_no, error)) = handle.get_pose() {
//! // `pose` is a 3x4 camera pose matrix for `page_no`.
//! }
//! ```
//!
//! For a runnable end-to-end version (including AR2 tracking), see the
//! `simple_nft` example; for the browser, see `crates/wasm`'s
//! `WasmKpmHandle`.
//!
//! ## Module layout
//!
//! | Module | Purpose |
//! |------------------|---------|
//! | [`backend`] | `FreakMatcherBackend` trait and associated error / data types |
//! | [`backend`] | [`FreakMatcherBackend`] trait and associated error / data types |
//! | [`handle`] | [`KpmHandle`] struct — central coordinator for KPM operations |
//! | [`types`] | Ported C structs (`KpmRefDataSet`, `KpmResult`, etc.) |
//! | [`cpp_backend`] | [`CppFreakMatcher`] — FFI backend (feature-gated) |
//! | [`types`] | Ported C structs ([`KpmRefDataSet`](types::KpmRefDataSet), etc.) |
//! | [`freak`] | The pure-Rust FREAK pipeline — detector, descriptor, pyramids, Hough voting, homography, `VisualDatabase` |
//! | [`rust_backend`] | [`RustFreakMatcher`] (default) and `DualFreakMatcher` (parity-asserting) |
//! | `cpp_backend` | `CppFreakMatcher` — FFI backend (only compiled with `--features ffi-backend`) |
//! | [`kpm_ffi`] | Raw bindgen bindings (feature-gated) |
//! | [`matching`] | [`MatchResult`](matching::MatchResult) wrapper |
//! | [`ref_data_set`] | [`RefDataSet`](ref_data_set::RefDataSet) collection |

// The KPM/NFT surface is the primary public API for library consumers, so it
// is held to full rustdoc coverage. This module tree is documented to 100%
// (#226); the warning is escalated to an error by CI's `clippy -D warnings`,
// so a new undocumented `pub` item here fails the build. The rest of the
// crate is still being back-filled — see #226 for the remaining modules.
#![warn(missing_docs)]

pub mod backend;
pub mod freak;
pub mod handle;
Expand Down
4 changes: 4 additions & 0 deletions crates/core/src/kpm/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -209,14 +209,18 @@ pub struct KpmResult {
/// 2D point with `f32` coordinates.
#[derive(Debug, Default, Clone, Copy, PartialEq)]
pub struct Point2f {
/// Horizontal position in pixels.
pub x: f32,
/// Vertical position in pixels.
pub y: f32,
}

/// 2D point with integer coordinates, used for corner detection.
#[derive(Debug, Default, Clone, Copy, PartialEq)]
pub struct Point2i {
/// Horizontal position in pixels.
pub x: i32,
/// Vertical position in pixels.
pub y: i32,
}

Expand Down
10 changes: 8 additions & 2 deletions crates/core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
//! - [`ar2::ar2_gen_feature_map`] — extracts gradient-based features at each pyramid level (`.fset`)
//! - [`ar2::AR2ImageSetT::save`] — writes the image pyramid as a JPEG-compressed `.iset` file
//! matching `ar2WriteImageSet()` format (~300 KB vs ~10 MB raw)
//! - [`kpm::KpmRefDataSet::generate`] — extracts FREAK descriptors for KPM recognition (`.fset3`)
//! - [`kpm::types::KpmRefDataSet::generate`] — extracts FREAK descriptors for KPM recognition (`.fset3`)
//!
//! See the `nft_marker_gen` example for a complete end-to-end usage.
//!
Expand Down Expand Up @@ -84,12 +84,18 @@
//! ### Recommended build for NFT marker generation (x86_64)
//!
//! ```bash
//! cargo run --release --features "ffi-backend,simd-x86-sse41,simd-x86-avx2" \
//! cargo run --release --features "log-helpers,simd-x86-sse41,simd-x86-avx2" \
//! --example nft_marker_gen -- --input marker.jpg --output marker --dpi 220
//! ```
//!
//! > **Always use `--release`** — debug builds are 5–10× slower.
//!
//! No C++ toolchain is required: since #179 `nft_marker_gen` produces the
//! complete marker set (`.iset` + `.fset` + `.fset3`) through the pure-Rust
//! [`RustFreakMatcher`](kpm::RustFreakMatcher). The `ffi-backend` feature is
//! **not** needed for marker generation — it exists only for development
//! cross-validation against the C++ backend.
//!
//! Detailed benchmark results can be found in `crates/core/benches/BENCHMARKS.md`.
//!
//!
Expand Down
Loading