feat(cs): ASAPv1 wire payload for Count Sketch

asapv1_golden/README.md

@@ -29,13 +29,21 @@ golden tests the **wire encoding**, isolated from the hash functions.
 | `hll_hip_p12` | HLL HIP, P12 | `01 03` | same registers + `hip_kxq0=1.5, hip_kxq1=2.5, hip_est=3.0` |
 | `cms_i64_regular_2x3` | Count-Min i64, RegularPath | `02 00` | 2×3 row-major `[[0,1,127],[128,300,65536]]` |
 | `cms_f64_fast_2x3` | Count-Min f64, FastPath | `02 00` | 2×3 row-major `[[0.0,1.5,2.25],[3.75,4.125,5.0625]]` |
+| `cs_i64_regular_2x4` | Count Sketch i64, RegularPath | `04 00` | 2×4 row-major `[[0,127,128,65536],[-1,-33,-32768,-2147483648]]` |
+| `cs_i64_fast_2x4` | Count Sketch i64, FastPath | `04 00` | same matrix — the pair differs only by `mode` |
 | `kll_f64_k200` | KLL f64, k=200 | `06 00` | integers `1..=50`, compaction seed 42 (recorded in metadata as `seed`) |
 | `kll_i64_k200` | KLL i64, k=200 | `06 00` | integers `1..=50`, compaction seed 42 (recorded in metadata as `seed`) |
 
-The i64 fixture deliberately spans the msgpack integer width boundaries
+The CMS i64 fixture deliberately spans the msgpack integer width boundaries
 (positive fixint / uint8 / uint16 / uint32) to lock the "non-negative integer →
 uint family, minimal width" rule (`docs/asapv1_wire_format.md` §5).
 
+The Count Sketch fixtures span those boundaries in **both** directions
+(negative fixint / int8 / int16 / int32 alongside the positive widths), because
+Count Sketch cells are signed — it adds `±weight`. The two files hold the same
+matrix and differ only by the `mode` metadata string, so the pair also pins that
+mode reaches the bytes.
+
 The KLL fixtures are a special case of "state is fixed, not hashed": KLL never
 hashes — it orders raw numeric values — so inserting `1..=50` places exactly
 those retained samples. `k=200` keeps the input below the level-0 capacity, so no

asapv1_golden/cs_i64_fast_2x4.hex

@@ -0,0 +1 @@
+4153415076310102040000000142000000168ab06d657461646174615f76657273696f6e01af686173685f70726f66696c655f6964bc70726f6a656374617361702e787868332e736565646c6973742e7631ae686173685f616c676f726974686dab787868335f36345f313238af736565645f64657269766174696f6eb4736565645f6c6973745f696e6465785f77726170ae696e7075745f656e636f64696e67b470726f6a656374617361702e696e7075742e7631a9736565645f6c697374dc0014cecafe3553cf000000ade3415118ce8cc70208ce2f024b2bce451a3df5ce6a09e667cebb67ae85ce3c6ef372cea54ff53ace510e527fce9b05688cce1f83d9abce5be0cd19cecbbb9d5dce629a292ace9159015ace152fecd8ce67332667ce8eb44a87cedb0c2e0db16d61747269785f736565645f696e64657800a4726f777302a4636f6c7304a46d6f6465a4666173749198007fcc80ce00010000ffd0dfd18000d280000000

asapv1_golden/cs_i64_regular_2x4.hex

@@ -0,0 +1 @@
+4153415076310102040000000145000000168ab06d657461646174615f76657273696f6e01af686173685f70726f66696c655f6964bc70726f6a656374617361702e787868332e736565646c6973742e7631ae686173685f616c676f726974686dab787868335f36345f313238af736565645f64657269766174696f6eb4736565645f6c6973745f696e6465785f77726170ae696e7075745f656e636f64696e67b470726f6a656374617361702e696e7075742e7631a9736565645f6c697374dc0014cecafe3553cf000000ade3415118ce8cc70208ce2f024b2bce451a3df5ce6a09e667cebb67ae85ce3c6ef372cea54ff53ace510e527fce9b05688cce1f83d9abce5be0cd19cecbbb9d5dce629a292ace9159015ace152fecd8ce67332667ce8eb44a87cedb0c2e0db16d61747269785f736565645f696e64657800a4726f777302a4636f6c7304a46d6f6465a7726567756c61729198007fcc80ce00010000ffd0dfd18000d280000000

docs/asapv1_wire_format.md

@@ -184,7 +184,7 @@ This registry is the master list of algorithms still to design payloads for.
 | `0x01 0x03` | HLL | HIP | Section 3.1 | implemented |
 | `0x02 0x00` | Count-Min | Count-Min | Section 3.2 | implemented |
 | `0x03 0x00` | Count-Min-with-heap (CMSHeap) | - | TBD | assigned in Go / payload not designed |
-| `0x04 0x00` | Count Sketch | - | TBD | assigned in Go / payload not designed |
+| `0x04 0x00` | Count Sketch | Count Sketch | Section 3.4 | implemented |
 | `0x05 0x00` | DDSketch | - | TBD | assigned in Go / payload not designed |
 | `0x06 0x00` | KLL | Compact | Section 3.3 | implemented |
 | `0x06 0x01` | KLL dynamic | Dynamic | Section 3.3 | implemented |
@@ -387,14 +387,27 @@ Both KLL variants (the compact fixed-buffer `KLL` and the growable `KLLDynamic`)
 
 **Item order (cross-language contract).** `levels` / `items` use the **top-most-level-first** layout, byte-for-byte matching `sketchlib-go`'s `KLLState`: index `i` in `levels` maps to compactor level `num_levels - 1 - i`, and level 0's run is in **input order**. The compact `KLL` grows its buffer leftward and stores level 0 reverse-input, so its encoder reverses level 0 back to input order (and its decoder reverses it in); `KLLDynamic` already stores this layout natively. Within a level, order past the first compaction is not guaranteed byte-identical across the two Rust variants (or across languages), but the retained set and quantiles agree — see the caveat on `KLL::wire_items`.
 
-### 3.4 onward: payloads not yet designed
+### 3.4: Count Sketch payload (`0x04 0x00`)
+
+Same shape as Count-Min (§3.2): the matrix dimensions (`rows` / `cols`) and the column-derivation **mode** (`"fast"`/`"regular"`) live in the metadata, so the payload is just the counters.
+
+| Pos | Field | Type | Notes |
+| ----- | ------- | ------ | ------- |
+| 0 | `counts` | array | packed **row-major**, `rows*cols` cells; `i64` elements, **signed** |
+
+The payload is a **1-element positional array `[counts]`**.
+
+There is **no `counter_type` metadata key**. Count Sketch counters must be signed and negatable, which leaves `i64` as the only wire type, so `kind_id` `0x04 0x00` already implies it (`i32` widens to `i64`; `i128` and exotic counters are not wire types — see Section 5, "Converting an exotic in-memory sketch"). This is the one metadata difference from Count-Min, whose counters may be `i64` or `f64`.
+
+Cells carry a sign: Count Sketch adds `±weight`, so a counter may be negative and a decoder must not assume monotonicity.
+
+### 3.5 onward: payloads not yet designed
 
 The remaining `kind_id`s reserve a family byte with payload TBD (Section 1 registry has their "assigned in Go" status). Likely shape when designed:
 
 | kind_id | Sketch | Likely payload |
 | --------- | -------- | --------- |
 | `0x03 0x00` | Count-Min-with-heap (CMSHeap) | similar to current CMS |
-| `0x04 0x00` | Count Sketch | similar to current CMS |
 | `0x05 0x00` | DDSketch | straightforward bucket |
 | `0x07 0x00` | Hydra-KLL | wraps KLL payloads (§3.3); nest one per counter |
 | `0x08 0x00` | SetAggregator | aggregation envelope, distinct from a stand-alone sketch (Section 1 mapping notes) |
@@ -510,6 +523,7 @@ Keep it through the transition, retire `portable` once goldens are in place.
 - **Q-CMS**: Count-Min is one `kind_id` (`0x02 0x00`); counter type and mode live in the metadata, so the id stays single.
 - **Q-KLL**: KLL metadata carries **no hash-spec group** — KLL is comparison-based and never hashes, so those fields have no truthful value. Its metadata is structural-only (`metadata_version`, `k`, `m`, `item_type`, optional `seed`) and is *not* `HashProfile`-derived. The two KLL variants (compact `0x06 0x00`, dynamic `0x06 0x01`) share one payload `[levels, items, coin]` and differ only by `kind_id`. `item_type` (`"f64"`/`"i64"`) is a metadata param, not a separate `kind_id` (mirrors Q-CMS's `counter_type`). Retained samples use the top-most-level-first layout that matches `sketchlib-go`'s `KLLState`.
 - **Q-KLL-SEED**: KLL records its reproducible compaction `seed` as an **optional** metadata key. It is construction config (so metadata, not payload, per the config→metadata rule), and it is the first optional key in v1: present only when the sketch carries a seed, omitted otherwise. Rationale: the payload's `coin` already carries the RNG's *current* position (enough to resume compaction), but `seed` is what a later `clear()` re-seeds from — so serializing it lets a decoded sketch keep `clear()`-reproducibility instead of falling back to wall-clock. Cost of omitting it is bounded (only a decoded-then-`clear()`ed sketch loses cross-run byte reproducibility — never correctness), but it is cheap to carry and future-proofs the checkpoint/restore path. `KLLDynamic` has no seed concept and never emits the key; the two variants are deliberately **not** forced to be symmetric here. Go carries and preserves the key without interpreting it.
+- **Q-CS**: Count Sketch is one `kind_id` (`0x04 0x00`) and mirrors Count-Min's metadata and `[counts]` payload with **one key removed: no `counter_type`**. Count Sketch counters must be signed and negatable, so `i64` is the only wire type and the `kind_id` already implies it — a key with a single legal value describes nothing. The variance from Q-CMS is deliberate: the two sketches are not forced symmetric where their type domains differ. Structural-param order: `... matrix_seed_index, rows, cols, mode`. Cells are signed, so a decoder must not assume monotonicity.
 - **Q-CMS-DIMS**: Count-Min `rows`/`cols` are **metadata** and the payload omits them. They are configuration that shapes the payload (like HLL's `precision`), so per the config-to-metadata rule they belong in the descriptor. The payload is then just `[counts]`. Canonical structural-param order: `... matrix_seed_index, rows, cols, counter_type, mode`.
 - **Q-VER**: no payload version field. A new incompatible encoding gets a **new `kind_id`**; retired ids are reserved forever and never recycled.
 - **Encoding**: metadata + payload are both msgpack; payload is a positional array. Byte-level rules in Section 4.

src/message_pack_format/native/countsketch.rs

@@ -1,15 +1,18 @@
 //! Native MessagePack codec impl for [`crate::sketches::countsketch::Count`].
-
-use serde::{Deserialize, Serialize};
+//!
+//! Only the canonical wire config — `i64` counters with a fast/regular mode
+//! (`CsWireMode`) — is serializable. Count Sketch counters must be signed and
+//! negatable, which leaves `i64` as the only wire-eligible type; exotic
+//! in-memory counters (i32/i128/…) must be converted first.
 
 use crate::message_pack_format::{Error, MessagePackCodec};
-use crate::sketches::countsketch::{Count, CountSketchCounter};
-use crate::{MatrixStorage, SketchHasher};
+use crate::sketches::countsketch::{Count, CsWireMode};
+use crate::{HashProfile, SketchHasher, Vector2D};
 
-impl<S, C, Mode, H: SketchHasher> MessagePackCodec for Count<S, Mode, H>
+impl<Mode, H> MessagePackCodec for Count<Vector2D<i64>, Mode, H>
 where
-    S: MatrixStorage<Counter = C> + Serialize + for<'de> Deserialize<'de>,
-    C: CountSketchCounter,
+    Mode: CsWireMode,
+    H: SketchHasher + HashProfile,
 {
     fn to_msgpack(&self) -> Result<Vec<u8>, Error> {
         Ok(self.serialize_to_bytes()?)

src/sketches/countsketch.rs

@@ -12,13 +12,13 @@ use crate::{
     FastPathHasher, FixedMatrix, MatrixFastHash, MatrixStorage, NitroTarget, QuickMatrixI64,
     QuickMatrixI128, RegularPath, SketchHasher, Vector2D, hash64_seeded,
 };
-use rmp_serde::{
-    decode::Error as RmpDecodeError, encode::Error as RmpEncodeError, from_slice, to_vec_named,
-};
 use serde::{Deserialize, Serialize};
 use std::marker::PhantomData;
 use std::ops::Neg;
 
+mod wire;
+pub(crate) use wire::CsWireMode;
+
 const DEFAULT_ROW_NUM: usize = 3;
 const DEFAULT_COL_NUM: usize = 4096;
 const LOWER_32_MASK: u64 = (1u64 << 32) - 1;
@@ -274,29 +274,11 @@ where
     }
 }
 
-// Serialization helpers for Count.
-impl<S, C, Mode, H: SketchHasher> Count<S, Mode, H>
-where
-    S: MatrixStorage<Counter = C> + Serialize,
-    C: CountSketchCounter,
-{
-    /// Serializes the sketch into MessagePack bytes.
-    pub fn serialize_to_bytes(&self) -> Result<Vec<u8>, RmpEncodeError> {
-        to_vec_named(self)
-    }
-}
-
-// Deserialization helpers for Count.
-impl<S, C, Mode, H: SketchHasher> Count<S, Mode, H>
-where
-    S: MatrixStorage<Counter = C> + for<'de> Deserialize<'de>,
-    C: CountSketchCounter,
-{
-    /// Deserializes a sketch from MessagePack bytes.
-    pub fn deserialize_from_bytes(bytes: &[u8]) -> Result<Self, RmpDecodeError> {
-        from_slice(bytes)
-    }
-}
+// `serialize_to_bytes` / `deserialize_from_bytes` live in the `wire` submodule:
+// they emit/read the ASAPv1 envelope (kind_id `0x04 0x00`) and are implemented
+// only for the wire-eligible config (`Vector2D<i64>` storage, fast/regular
+// mode, a profiled hasher). Exotic counters (i32/i128/…) and non-`Vector2D`
+// storage must be converted first — see `docs/asapv1_wire_format.md` §5.
 
 // Regular-path Count operations.
 impl<S, C, H: SketchHasher> Count<S, RegularPath, H>
@@ -1080,24 +1062,6 @@ mod tests {
         );
     }
 
-    #[test]
-    fn count_sketch_round_trip_serialization() {
-        let mut sketch = Count::<Vector2D<i32>, RegularPath>::with_dimensions(3, 8);
-        sketch.insert(&DataInput::U64(42));
-        sketch.insert(&DataInput::U64(7));
-
-        let encoded = sketch.serialize_to_bytes().expect("serialize Count");
-        assert!(!encoded.is_empty());
-        let data_copied = encoded.clone();
-
-        let decoded = Count::<Vector2D<i32>, RegularPath>::deserialize_from_bytes(&data_copied)
-            .expect("deserialize Count");
-
-        assert_eq!(sketch.rows(), decoded.rows());
-        assert_eq!(sketch.cols(), decoded.cols());
-        assert_eq!(
-            sketch.as_storage().as_slice(),
-            decoded.as_storage().as_slice()
-        );
-    }
+    // Round-trip serialization moved to the `wire` submodule's tests: the wire
+    // config is `Vector2D<i64>` (this test used the non-wire i32 counter).
 }