Skip to content

V0.3.1/woql dsl v0 2 - #31

Merged
thanos merged 3 commits into
mainfrom
v0.3.1/WOQL_DSL_v0_2
Jun 25, 2026
Merged

thanos merged 3 commits into
mainfrom
v0.3.1/WOQL_DSL_v0_2

Conversation

@thanos

@thanos thanos commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Overview

v0.3.1 brings the WOQL DSL from a 7-operator v0.1 subset to a fully usable
~70-operator surface (Tier 1+2), matching the Python/JS clients' vocabulary.
This release introduces a wire-correct 4-wrapper value model, a dual-mode path
DSL (string-compiled parser + structured builders), and full round-trip
serialization for every operator.

The encoder was refactored to match the canonical WOQL JSON-LD wire format,
fixing two latent bugs in the v0.3 encoder (read_document field ordering and
triple object encoding).

What's new

WOQL DSL v0.2 (ADR-0008)

Expanded from 7 to ~70 operators across all major families:

  • Logical combinators (4): not_/1, opt/1 (alias optional/1),
    once/1, immediately/1.
  • Query modifiers (9): distinct/2, limit/2, start/2, order_by/2
    (accepts tuple-list [{"v:Name", :asc}] or keyword-list [name: :asc]),
    group_by/4, count/2, collect/3, star/0, all/0.
  • Graph patterns (12): quad/4, added_triple/3, removed_triple/3,
    added_quad/4, removed_quad/4, add_triple/3, delete_triple/3,
    add_quad/4, delete_quad/4, update_triple/3, update_quad/4.
  • Comparison (5): less/2, greater/2, gte/2, lte/2, like/3.
  • Schema ops (3): isa/2, sub/2 (alias subsumption/2), cast/3
    (alias typecast/3).
  • Arithmetic (9): eval/2, plus/1, minus/1, times/1, divide/1,
    div/1, exp/2, floor/1, sum/2.
  • String ops (10): concat/2, join/3, substr/5, trim/2, upper/2,
    lower/2, pad/4, split/3, length/2, regexp/3.
  • List/Set/Dict (8): dot/3, member/2, slice/4, set_difference/3,
    set_intersection/3, set_union/3, set_member/2, list_to_set/2.
  • Path/navigation (1): path/3..4 with dual-mode DSL.
  • ID generation (3): unique/3, idgen/3, idgen_random/2.
  • Documents (3): insert_document/1, update_document/1,
    delete_document/1.
  • Graph context (4): using/2, from/2, into/2, comment/2.
  • Graph meta (2): size/2, triple_count/2.
  • Literal/value helpers (8): var/1, iri/1, string/1, boolean/1,
    datetime/1, date/1, literal/2, true_/0.

Dual-mode Path DSL

Path queries accept both string patterns and structured builders:

# String pattern — compiled via tokenizer
WOQL.path("v:S", "friend*{1,3}", "v:O")

# Structured — idiomatic Elixir, no parser
WOQL.path("v:S",
  WOQL.Path.path_star(WOQL.Path.path_pred("friend")),
  "v:O"
)

String pattern grammar: predicates (friend), inverse (<friend), star
(*), plus (+), bounded repetition ({n}, {n,m}, {n,}), alternation
(|), sequencing (,), any (.), and grouping ((...)).

4-wrapper value model

The JSON-LD encoder now uses four value-wrapper types matching the Python/JS
clients:

Wrapper Used for
NodeValue Subjects, predicates, identifiers, IRIs
Value Triple objects, comparison operands, type_of, read_document
DataValue String-op operands, ID-gen keys
ArithmeticValue Arithmetic operands

Module split

woql.ex split into sub-modules following the Client.Params pattern:

lib/terminus_db/woql.ex           # public API, execute, to_jsonld/from_jsonld
lib/terminus_db/woql/encoder.ex   # 4-wrapper JSON-LD encoder
lib/terminus_db/woql/decoder.ex   # JSON-LD → Query struct
lib/terminus_db/woql/path.ex      # path pattern parser + structured builders
lib/terminus_db/woql/literal.ex   # value/literal helpers

Tests

  • 442 unit tests + 119 doctests + 9 properties, 0 failures.
  • 27 integration tests (against live TerminusDB 12, excluded from default run).
  • 94.0% coverage (enforced minimum 80%).
  • Property tests: random query trees round-trip through to_jsonld ∘ from_jsonld.
  • Every operator has builder, encoder, decoder, and round-trip tests.

Breaking changes

  1. triple/3 object encoding: constant string objects now encode as
    Value with xsd:string data (literals), matching Python. Previously they
    encoded as NodeValue with node (IRIs). Migration: pass
    WOQL.iri("...") explicitly when an IRI object is intended.

  2. read_document/2 field ordering: the document id is now under
    "identifier" (NodeValue) and the output variable under "document"
    (Value), matching the canonical wire format. Previously these were swapped.

  3. eq/2 operands: now wrapped in Value (was DataValue).

  4. type_of/2: both value and type now use Value (was
    NodeValue for type).

Bug fixes

  • read_document/2: field ordering now matches the canonical WOQL JSON-LD
    wire format (was reversed, causing incorrect encoding).

thanos added 3 commits June 25, 2026 16:07
Category	Operators
Logical combinators	not_, opt, once, immediately
Query modifiers	distinct, limit, start, order_by (both forms), group_by, count, collect, star, all
Graph patterns	quad, added/removed_triple/quad, add/delete_triple/quad, update_triple/quad
Comparison	less, greater, gte, lte, like
Schema ops	isa, sub, cast
Arithmetic	eval, plus, minus, times, divide, div, exp, floor, sum
String ops	concat, join, substr, trim, upper, lower, pad, split, length, regexp
List/Set/Dict	dot, member, slice, 5 set ops
Path/navigation	path/3..4 (string parser + structured builders)
ID generation	unique, idgen, idgen_random
Documents	insert_document, update_document, delete_document
Graph context	using, from, into, comment
Graph meta	size, triple_count
Literal helpers	var, iri, string, boolean, datetime, date, literal, true_
Total new
Plus v0.1 existing	triple, and_, or_, eq, select, read_document, type_of
Grand total
Key architecture changes:
- 4-wrapper value model (NodeValue, Value, DataValue, ArithmeticValue)
- read_document field ordering fixed
- triple string objects → xsd:string literals (use iri/1 for IRIs)
- Module split: woql.ex + encoder.ex + decoder.ex + path.ex + literal.ex
- Dual path DSL: string-compiled parser + structured builders
- order_by accepts both tuple-list and keyword-list forms
…atchError; also handles leftover tokens

F2	Path quantifier {n} now correctly produces to = n (exactly n), {n,} produces to = nil (unbounded), {n,m} produces to = m (bounded). Restored :comma token in tokenizer; commas act as sequence separators outside quantifiers
D1	Fixed CHANGELOG arities: insert_document/1, update_document/1, star/0, all/0, true_/0
D2	Guide type_of comment: DataValue → Value
D3	Guide struct name: %WOQL.Query{} → %TerminusDB.WOQL{}
D4	Guide write example: triple("v:New", "rdf:type", "@Schema:Person") → triple("v:New", "rdf:type", iri("@Schema:Person"))
T1	insert_document integration test now passes an actual document map (%{"@type" => "Person", "name" => "Bob", "age" => 35}) instead of WOQL.string("Person/Bob")
ID	Fix
T2	Schema now uses @key: Lexical so document IDs are deterministic (Person/Alice); delete_document test verifies the document is actually gone
D5	Added pad/4 and split/3 to guide string ops table
T3	Integration tests now verify result content: order_by checks sorted order, less checks Alice/Bob in and Carol out, count checks count == 2, limit checks count == 2, add_triple checks the triple was written
T4	Property test generator extended from ~17 to ~45 operators (added arithmetic, string, schema, set, document, graph context/meta ops)
MT1	Added update_triple round-trip-as-and_ test
MT4	Added malformed pattern tests: empty string, unbalanced parens, invalid quantifier, trailing pipe
ID	Fix
Q1	execute/3 now returns {:error, %Error{reason: :config}} instead of raising; added :config to Error reason type
Q2	update_triple/3 and update_quad/4 @doc now warns about the "v:OldObject" internal variable collision risk
T5	Path integration test now inserts actual Knows edges and uses target+ to verify traversal finds connected nodes
Quality gate results
… encoded as DictionaryTemplate with FieldValuePair entries (matching JS doc() wrapper). insert_document/2 and update_document/2 now accept an optional identifier argument (variable bound to inserted doc's IRI).

 - add_triple — instance_not_cardinality_one for age	Added age triple (WOQL.literal(42, "integer")) to satisfy schema's cardinality-one constraint. Also reordered rdf:type first.
 - delete_document — Access.get/3 on string	Fixed Document.get(cfg, "Person/Alice") → Document.get(cfg, id: "Person/Alice") (keyword list, not positional).
 - path — no results for target+	Fixed path pattern from target+ to (<source,target)+ — Person documents don't have outgoing target edges; need to traverse backwards through source to Knows, then forwards through target to Person.
 - read_document — Access on list with "@id"	Fixed doc_id extraction to handle list-of-strings response from Document.insert.
@thanos
thanos merged commit 9d813fd into main Jun 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant