quantcpp 0.8.2: add quant_free_string, eliminate ask() leak

New public C API export in quant.h:

  void quant_free_string(char* str);

The implementation lives in the same translation unit as quant_ask, so
its free() call uses the dylib's own malloc zone — no cross-heap abort
on macOS arm64 / Windows. Python ctypes wrapper calls it instead of the
v0.8.1 skip-and-leak workaround.

Backwards compat: bindings use hasattr(lib, 'quant_free_string') so
older loaded single-headers still work (with the old leak behavior).
New PyPI installs ship the updated quant.h.

Verified: 3x consecutive Model.ask() calls clean exit under faulthandler.

Honest correction track now at 7 (all self-found, all logged).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: unamedkr

CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## [0.8.2] — 2026-04-09 (quant_free_string + leak fix)
+
+### Eliminated the v0.8.1 leak in `Model.ask()`
+
+v0.8.1 worked but leaked ~65 KB per `ask()` call, because the Python wrapper couldn't safely call `libc.free()` on a pointer allocated inside `libquant.dylib`'s malloc heap (cross-zone abort on macOS arm64).
+
+v0.8.2 adds a tiny new export to the public C API:
+
+```c
+// quant.h
+void quant_free_string(char* str);
+```
+
+The implementation lives in the same translation unit as `quant_ask`, so its `free()` call uses the dylib's malloc zone — same heap, no abort. The Python wrapper now calls `lib.quant_free_string(ptr)` instead of skipping the free.
+
+Backwards compat: the binding uses `hasattr(lib, 'quant_free_string')` so older single-headers loaded via `QUANTCPP_LIB=...` continue to work (with the old leak behavior). New installs from PyPI 0.8.2 ship the updated `quant.h`.
+
+Verified: `Model("model.gguf").ask("hi")` × 3 in a row, clean exit, no abort, no leak warning under faulthandler.
+
+### Honest correction track is now 7 (still all self-found)
+
+This is the 7th correction logged in v0.6.x → v0.8.x. Found by running `Model.ask` repeatedly in the v0.8.1 verification cycle. Goal: stay 100% self-found before any external user reports a regression.
+
+### Still pending in v0.8.x
+
+- **`kv_compress=1` / `=2` re-enable in Python bindings** — still requires `quant.h` regeneration against the v0.8.0+ multi-file source (the bundled header is an Apr-6 snapshot whose UNIFORM_4B path aborts on Llama). Tracked in [Issue #18](https://github.com/quantumaikr/quant.cpp/issues/18). Will land when we regenerate the single header.
+
+---
+
 ## [0.8.1] — 2026-04-09 (Python bindings hotfix)
 
 ### `pip install quantcpp` is now actually usable

bindings/python/pyproject.toml

@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "quantcpp"
-version = "0.8.1"
+version = "0.8.2"
 description = "Single-header LLM inference engine with KV cache compression (7× compression at fp32 parity)"
 readme = "README.md"
 license = { text = "Apache-2.0" }

bindings/python/quantcpp/__init__.py

@@ -21,7 +21,7 @@
     from importlib.metadata import version as _pkg_version
     __version__ = _pkg_version("quantcpp")
 except Exception:
-    __version__ = "0.8.1"  # fallback for editable / source-tree imports
+    __version__ = "0.8.2"  # fallback for editable / source-tree imports
 
 import os
 import threading
@@ -159,13 +159,11 @@ def ask(self, prompt: str) -> str:
         result = ctypes.cast(ptr, ctypes.c_char_p).value
         text = result.decode("utf-8", errors="replace") if result else ""
 
-        # NOTE (v0.8.1): the C string returned by quant_ask is allocated
-        # inside libquant.dylib's malloc heap. Calling ctypes.CDLL(None).free
-        # on it crashes on macOS arm64 because Python's libc handle resolves
-        # to a different malloc zone than the dylib's. We accept a ~65 KB
-        # leak per ask() call as a temporary tradeoff. quant_free_ctx /
-        # quant_free_model release the bulk of the memory at end of session.
-        # Tracked: add quant_free_string(void*) to quant.h in v0.8.2.
+        # Free via the dylib's own free wrapper (added in v0.8.2). Falls back
+        # to a leak if the loaded library is an older single-header that
+        # doesn't export quant_free_string — preserves binary compat.
+        if hasattr(lib, "quant_free_string"):
+            lib.quant_free_string(ptr)
 
         return text
 

bindings/python/quantcpp/_binding.py

@@ -134,6 +134,13 @@ def _setup_signatures(lib: ctypes.CDLL) -> None:
     lib.quant_ask.argtypes = [ctypes.c_void_p, ctypes.c_char_p]
     lib.quant_ask.restype = ctypes.c_void_p  # We use c_void_p so we can free()
 
+    # void quant_free_string(char*) — added in v0.8.2 to free quant_ask
+    # results without cross-heap libc.free() crashes on macOS arm64.
+    # Optional: older single-headers may not export this symbol.
+    if hasattr(lib, "quant_free_string"):
+        lib.quant_free_string.argtypes = [ctypes.c_void_p]
+        lib.quant_free_string.restype = None
+
     # void quant_free_ctx(quant_ctx* ctx)
     lib.quant_free_ctx.argtypes = [ctypes.c_void_p]
     lib.quant_free_ctx.restype = None

quant.h

@@ -58,6 +58,12 @@ int quant_generate(quant_ctx* ctx, const char* prompt,
 // Generate and return full response as string. Caller must free().
 char* quant_ask(quant_ctx* ctx, const char* prompt);
 
+// Free a string returned by quant_ask. Always use this — never call libc
+// free() on the returned pointer directly. The string lives in the dylib's
+// malloc heap, which on macOS arm64 / Windows can be a different malloc
+// zone than the caller's libc.
+void quant_free_string(char* str);
+
 // Free resources.
 void quant_free_ctx(quant_ctx* ctx);
 void quant_free_model(quant_model* model);
@@ -15770,6 +15776,13 @@ char* quant_ask(quant_ctx* ctx, const char* prompt) {
     return output;
 }
 
+void quant_free_string(char* str) {
+    /* The string was malloc()'d inside this translation unit (quant_ask),
+     * so it must be free()'d here too — same malloc zone, no cross-heap
+     * crash on macOS arm64 / Windows. */
+    if (str) free(str);
+}
+
 void quant_free_ctx(quant_ctx* ctx) {
     if (!ctx) return;
     tq_free_state(ctx->state);