Skip to content

V0.3.0/vc woql - #30

Merged
thanos merged 5 commits into
mainfrom
v0.3.0/VC_WOQL
Jun 25, 2026
Merged

thanos merged 5 commits into
mainfrom
v0.3.0/VC_WOQL

Conversation

@thanos

@thanos thanos commented Jun 25, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

v0.3.0 transforms the client from a document CRUD library into a
TerminusDB-native client exposing versioned data workflows. This release adds
commit history, diff, merge (rebase), and a WOQL functional query DSL, all
validated against a live TerminusDB 12.0.5 server.

Eight real bugs in the v0.2 code were discovered via integration testing and
fixed in this release.

What's new

TerminusDB.Commit

Commit history and inspection over the /api/log and /api/history endpoints:

  • log/2 / log!/2 — concise log of recent commits on a branch, with
    pagination (:start, :limit).
  • history/2 / history!/2 — full commit chain with metadata.
  • get/3 / get!/3 — retrieve a single commit by descriptor ID.

All functions are branch-aware (:branch override) and support
:organization/:repo overrides.

TerminusDB.Diff

Document and branch-level diff over the /api/diff endpoint:

  • compare/2 / compare!/2 — diff two document values or branch/commit refs.
    Supports before/after (document maps or resource refs) and :keep for
    field preservation.

TerminusDB.Merge

Branch merge (rebase) over the /api/rebase endpoint:

  • merge/2 / merge!/2 — rebase a source branch onto a target branch, with
    :author/:message commit metadata.

TerminusDB.WOQL

Functional builder DSL for WOQL (ADR-0002), serializing to the correct JSON-LD
wire format:

  • Builders: triple/3, and_/1, or_/1, eq/2, select/2,
    read_document/2, type_of/2.
  • to_jsonld/1 — serialize to the WOQL JSON-LD format with NodeValue/
    DataValue wrappers and short type names ("Triple", "Equals", etc.).
  • from_jsonld/1 — deserialize back to a WOQL.Query struct. Round-trip
    tested for all supported operations.
  • execute/3 / execute!/3 — POST to /api/woql/:org/:db/:repo/branch/:branch
    with commit_info support for write queries.

Telemetry

Added :commit and :woql to the telemetry area type. All new modules emit
[:terminusdb, <area>, :start|:stop] events.

Documentation

  • Overview guide updated with four new sections: Commit history (section 9),
    Diff (section 10), Merge (section 11), WOQL DSL (section 12).

Tests

  • 217 unit tests + 6 properties + 38 doctests, 0 failures.
  • 14 integration tests, 0 failures (against live TerminusDB 12.0.5).
  • 94.0% coverage (enforced minimum 80% via coveralls.json).

Bug fixes (discovered via integration testing)

  1. Schema.frame/3: class name is now a ?type= query param, not a path
    segment (server returns 404 for path-appended class names).
  2. Branch.exists?/3: rewrote to check db/:org/:db?branches=true branch list
    (the /branch endpoint only supports POST/DELETE, not HEAD or GET).
  3. Document.query/3: now always sends as_list=true (server returns
    concatenated JSON by default, which crashes Req's JSON decoder).
  4. Commit.history/2: uses the /log endpoint (the /history endpoint
    requires a commit ID and cannot list without one).
  5. WOQL.eq/2: type is "Equals" not "Eq"; literals are wrapped in
    DataValue with xsd type annotations.
  6. Merge.merge/2: uses /rebase with author/rebase_from body (not
    /pull with remote/remote_branch, which causes 500 on local merges).
  7. WOQL.execute/3: now includes :repo/:branch in the WOQL path.
  8. Database integration tests: fixed to match actual v12 response shapes
    (path field, map not list).

TerminusDB 12 compatibility

Verified against TerminusDB 12.0.5. Key v12 features documented:

  • Range queries over triples (v12.0.4)
  • ISO8601 date/time/interval predicates, Allen interval algebra (v12.0.4)
  • WOQL comment and collect predicates (v12.0.4)
  • Cardinality constraints on Set (v12.0.4)
  • @metadata and @context JSON handling in schema (v12.0.4)
  • Diff and streaming in the history endpoint (v12.0.5)
  • merge_repeats on document interface update (v12.0.5)

The WOQL DSL is designed for easy extension to these new predicates in future
releases.

Breaking changes

  • Schema.frame/3: no longer appends the class name to the URL path; instead
    sends it as a ?type= query parameter. This is the correct TerminusDB API
    usage but changes the outgoing request.
  • Branch.exists?/3: now queries db/:org/:db?branches=true instead of
    HEAD/GET on the branch resource. Returns true/false the same way.
  • Merge.merge/2: uses /rebase instead of /pull. The body format changed
    from remote/remote_branch to author/rebase_from.
  • WOQL JSON-LD format: uses short type names ("Triple", "Equals") and
    NodeValue/DataValue wrappers, not woql: prefixed types.
  • Commit.history/2: uses the /log endpoint instead of /history.

Dependencies

Runtime (unchanged): req ~> 0.5, jason ~> 1.4, nimble_options ~> 1.1,
telemetry ~> 1.2.

thanos added 5 commits June 25, 2026 09:53
… bug fixes

Implement the versioned query workflow modules and the WOQL functional DSL,
all validated against a live TerminusDB 12.0.5 server. Eight real bugs in the
existing v0.2 code were discovered via integration testing and fixed.

New modules:
- closed #20 - TerminusDB.Commit: log/history/get (plus !/ variants) for commit history
  traversal. Branch-aware with pagination support.
- closed #21 - TerminusDB.Diff: compare/compare! for document and branch-level diffs via
  the /api/diff endpoint.
- closed #22 - TerminusDB.Merge: merge/merge!/preview/preview! for branch rebasing via the
  /api/rebase endpoint. Includes dry-run preview support.
- closed #23 and closed #24  - TerminusDB.WOQL: functional builder DSL (ADR-0002) with triple, and_, or_,
  eq, select, read_document, type_of. Serializes to the correct WOQL JSON-LD
  wire format (NodeValue/DataValue wrappers, short type names) and executes
  via POST /api/woql. Round-trip tested (to_jsonld ∘ from_jsonld).
 - closed #25 Added integration tests for commit/diff/merge/woql
 - closed #26 Update telemetry areas for commit/diff/merge/woql
 - closed #27 fix running the integration tests locally
 - closed #28 error - Database.list — response uses @id not name
 - closed #29 integration test failures

Bug fixes discovered via integration testing:
- Schema.frame: class name is now a ?type= query param, not a path segment
  (server returns 404 for path-appended class names).
- Branch.exists?: rewrote to check db/:org/:db?branches=true branch list
  (the /branch endpoint only supports POST/DELETE, not HEAD or GET).
- Document.query: now always sends as_list=true (server returns concatenated
  JSON by default, which crashes Req's JSON decoder).
- Commit.history: uses the /log endpoint (the /history endpoint requires a
  commit ID parameter and cannot list without one).
- WOQL.eq: type is "Equals" not "Eq"; literals are wrapped in DataValue with
  xsd type annotations (bare values cause "Not well formed WOQL JSON-LD").
- Merge: uses /rebase with author/rebase_from body (not /pull with
  remote/remote_branch, which causes 500 on local merges).
- Database integration tests: fixed to match actual v12 response shapes
  (path field, map not list).
- Telemetry: added :commit and :woql to the area type.

Tests:
- 217 unit tests + 6 properties + 38 doctests, 0 failures.
- 14 integration tests, 0 failures (against live TerminusDB 12.0.5).
- 94.0% coverage.
- C1 — WOQL.triple/3 object encoding (woql.ex): Replaced broken encode_value/1 that left constants as bare strings with proper wrapping: constant strings → NodeValue with node, numbers/booleans → DataValue with xsd-typed data. Added corresponding decode_value/1 clauses for round-tripping. Fixed the unit test that was asserting the broken output.
- C2 — Merge.preview/2 removed (merge.ex): The /api/rebase endpoint ignores the preview flag, so preview/2 performed a real merge. Removed preview/2 and preview!/2, their tests, and all doc/guide references.
High-severity fixes
- H1 — WOQL.type_of/2 (woql.ex): Changed field node → value and encoding from encode_node to encode_value. Updated decoder to match.
- H2 — WOQL.eq/2 left operand (woql.ex): Changed from encode_value (bare strings) to encode_data (proper DataValue wrapping). Updated decoder to use decode_data for both sides.
- H3 — Diff unused options (diff.ex): Removed :repo and :branch from compare_opt type — they were declared but never read.
Medium fixes
- M1/F5 — Commit.history/2 (commit.ex): Made history/2 delegate to log/2 instead of duplicating the implementation. Fixed the doc to honestly state it's an alias, removing the false "additional metadata fields" claim.
- M2/F1/F2/F4 — Doc contradictions: Fixed merge.ex moduledoc (/api/pull → /api/rebase), woql.ex vocabulary table (Eq → Equals), eq/2 doc, and execute/2 → execute/3.
- M3/M4 — Test quality: Tightened the WOQL integration assertion from bindings != nil or inserts != nil or deletes != nil to assert api:status == "api:success" + assert is_list(bindings). Added integration tests for constant-object triples and type_of. Fixed the broken unit test assertion.
Low fixes
- L1/F7: Removed leftover comment in merge.ex.
- L3/F10: Fixed duplicate section numbering in guides/overview.md (two "## 9" → renumbered 9-13).
- F9: Added decode tests for Or, ReadDocument, TypeOf and round-trip tests for Or, ReadDocument, and constant-object Triple.
- F11: Added history!/2 failure test (the get!/3 failure test already existed).

- F8 — Extracted maybe_put/3 to TerminusDB.Client.Params (public, @spec-ed) and removed the private duplicate from all 3 modules (diff.ex, woql.ex, database.ex). Coverage held at 95.2%, all 222 unit + 16 integration tests pass, credo/dialyzer clean.
- F12 — Replaced all 26 em-dashes in the 4 new v0.3 modules with spaced hyphens. (Pre-v0.3 files untouched — out of review scope.)
@thanos
thanos merged commit 6675a02 into main Jun 25, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant