Skip to content

feat: add selectable phone call voice stacks - #83

Merged
dimavrem22 merged 39 commits into
mainfrom
plan/phone-call-voice-stack
Aug 1, 2026
Merged

feat: add selectable phone call voice stacks#83
dimavrem22 merged 39 commits into
mainfrom
plan/phone-call-voice-stack

Conversation

@dimavrem22

@dimavrem22 dimavrem22 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Executive Summary

Adds explicit phone-call voice-stack selection and reliable Inkbox Voice AI post-call follow-up to the Hermes Inkbox plugin.

  • Offers Inkbox Voice AI, OpenAI Realtime API, and Inkbox TTS/STT through Hermes-native setup controls.
  • Makes inbound routing, outbound calls, and gateway reconciliation follow one persisted voice-stack choice.
  • Proves promised hosted-call SMS follow-up through durable tool lifecycle records and exact API-side effects.

Description

The setup wizard persists one canonical voice stack, validates Realtime credentials before saving them, supports contact-scoped and YOLO Voice AI authority, and keeps administrative credentials transient. Outbound tools use hosted calls for Inkbox Voice AI and the client media bridge for Realtime or TTS/STT. Voice AI calls omit a per-call authority override so the API applies the saved identity default. Existing installations without the new setting retain legacy Realtime auto-detection.

The gateway registers call lifecycle, iMessage, and agent-to-agent events in separate identity-owned subscription rows at the same canonical receiver URL. Hosted completion events fetch the authoritative transcript, include open actions and the current remote phone number, enqueue a suppressed-text Hermes reconciliation turn, and persist bounded receipts for deduplication and crash recovery.

For a required post-call SMS, the plugin binds trusted call context to the real Hermes session. pre_tool_call durably records a sanitized pending attempt before inkbox_send_sms can perform an external write; post_tool_call replaces that exact attempt by hashed tool_call_id with its final sanitized result. Settlement requires exactly one successful tool call to the authoritative remote number. A missing initial call or deterministic pre-send argument/content error gets exactly one mandatory correction turn. Wrong recipients, duplicate calls, timeouts, rate limits, 5xx responses, carrier/consent failures, unknown results, pending crash-window writes, and any unsuccessful correction are terminal and never automatically replayed.

Open post-call action items remain authoritative. Transcript fallback uses clause-aware matching and requires explicit post-call timing plus an explicit text/SMS action. It rejects generic live-call requests, noun/past references, email sends, and negated instructions. This avoids forcing a duplicate or forbidden SMS when Voice AI may already have handled the request during the call.

If trusted session lookup or context persistence fails, the call/attempt is terminal and pre_tool_call blocks the affected target before an external write. Failure markers contain only target digests and do not block unrelated recipients. Durable context files are mode 0600 inside a mode 0700 directory, and temporary JSON files are created mode 0600 before content is written. Settlement state excludes SMS bodies, raw tool results, transcripts, contact memories, and credentials. Terminal cleanup removes per-call observations and queued/active session context.

Reason

Inferring voice behavior from credential presence allowed setup, inbound routing, outbound tools, and post-call work to disagree. Separately, a successful model turn could return [SILENT] without executing a promised post-call SMS, while blind restart replay could duplicate an external write after a crash. The plugin needs one explicit runtime choice and host-native side-effect settlement so promised work is either proven once or fails closed.

Decisions

  • Canonical stack: INKBOX_VOICE_STACK selects inkbox_voice_ai, openai_realtime, or inkbox_tts_stt for all call paths.
  • Wire compatibility: User-facing copy says Inkbox Voice AI while hosted_agent remains the API transport value.
  • Authority: Hosted outbound calls omit a per-call override and inherit the saved administrator-approved identity setting.
  • Credential isolation: Setup may reuse an administrative credential during configuration but never persists it as the gateway key.
  • Side-effect source of truth: Hermes pre_tool_call and post_tool_call hooks, keyed to the real session_id and hashed tool_call_id, determine SMS settlement.
  • Bounded correction: Exactly one correction is allowed only after no initial tool call or a deterministic pre-send failure. [SILENT] is forbidden on that mandatory correction; an unsuccessful correction is terminal and may settle silently without a third attempt.
  • Restart safety: Clean turns with no tool attempt may replay. Pending or ambiguous attempts never replay; observed success completes; deterministic attempt-one failure resumes directly as the one correction.
  • Fail closed: Session lookup or persistence failure terminalizes the affected call and blocks only its target before send.
  • Recipient identity: Post-call sends use the event's authoritative remote phone number; generated memory cannot replace it.
  • Realtime ordering: Tool-result responses wait for the active Realtime response to finish.
  • Voicemail policy: Product calls retain detection enabled by default while every call-capable live CI request explicitly disables it and verifies the persisted call record.

Testing

Local verification

  • Full offline suite: 486 passed, 25 live/host-dependent skipped.
  • Focused wizard, voice, hosted completion, and call policy suites: 164 passed.
  • Contract suite against the current Hermes host checkout: 12 passed.
  • ruff check ., python -m compileall, uv lock --check, and git diff --check: passed.
  • Workflow contract audit proves every agent-capable live lane, including A2A, disables voicemail detection. The hosted-call driver now stays connected until pytest observes both caller intent and the matching open action, uses an exact five-word distinct everyday-word marker that stays within the action summarizer's reliable span, and leaves the early hangup to the acceptance test.
  • Plugin metadata is bumped from 0.2.7 to 0.2.8 and all runtime, lock, workflow, and manual-playground references use published Inkbox SDK 0.5.9.

Live-test inventory

  • New — tests/live/test_voice.py::test_outbound_call_inkbox_voice_ai_and_completion: proves a current hosted call uses hosted_agent, exactly matches the saved authority, persists a reason, disables voicemail detection, records both caller intent and an open SMS action before test-owned hangup, receives call.ended, settles against the authoritative caller, creates exactly one fresh sender-side SMS row, and remains duplicate-free through the full grace period. It excludes baseline IDs, requires server creation watermarks, enforces driver/AUT timestamp skew, and observes the exact pair for a duplicate grace before accepting it. The exact five-word everyday-word marker survives TTS/PSTN/STT while avoiding conventional phonetic-alphabet substitutions and a known six-word action-summary truncation; partial markers still fail.
  • Updated — setup wizard contracts: proves reruns default to each valid saved stack, invalid fallback selections reprompt, cancellation exits cleanly, and an agent-scoped credential entered for YOLO is rejected before any authority mutation or persistence and returns to all three stack choices.
  • Updated — authoritative hosted callback settlement: injects a similarly named contact and conflicting remembered phone number, then proves a model attempt to send there is terminal and nonretryable rather than redirecting or retrying the current call's authoritative recipient.
  • Updated — hosted action request contract: repeated live runs showed that a generic “SMS containing these words” request could reach the caller transcript while the hosted action summarizer omitted or reinterpreted the marker. The driver now requires Voice AI to create the post-call action immediately with the exact SMS body and read back all five words only after the action is saved. A deterministic workflow contract locks that wording; transcript, durable action, exact recipient, exact body, and duplicate gates remain strict.
  • Updated — hosted speech marker: the deterministic run marker now draws from 20 distinct common multisyllabic words selected for clearly separated pronunciations. This avoids conventional phonetic-alphabet phrase bias and short-word substitutions across the hosted audio path while preserving five-word current-run entropy, position offsets, uniqueness, and full exact matching at every transcript/action/SMS gate.
  • Updated — hosted SMS settlement acceptance: matches the complete contiguous speech-normalized run marker within the current SMS rather than incorrectly requiring the marker to be the entire SMS body. It retains authoritative-recipient filtering, baseline IDs, the server timestamp watermark, completed receipt, exactly-one result, and the full duplicate grace. Bounded diagnostics expose only candidate counts and booleans, and assertions no longer print SMS content. This closes the false negative where Hermes delivered and reconciled the requested marker with harmless surrounding prose but the live harness polled until timeout.
  • Updated — tests/live/test_voice.py::test_inbound_call_inkbox_tts_stt: refreshes the exact placed call through the API and verifies persisted voicemail_detection=disabled. This closes the gap where the request argument was asserted without proving what the server stored.
  • Updated — tests/live/test_voice.py::test_outbound_call_realtime: correlates the fresh AUT-owned outbound call and verifies persisted disabled voicemail detection. This closes false matches against older or opposite-side call records.
  • Updated — tests/live/test_voice.py::test_outbound_call_realtime_direct_contact_lookup: applies the same exact-call persisted voicemail proof to Hermes' direct-contact Realtime flow. This preserves the host-specific tool path while enforcing the fleet call policy.
  • Updated — tests/live/test_cross_channel.py::test_email_request_gets_call: snapshots both identities, requires the fresh driver inbound and AUT outbound legs, and verifies voicemail detection on the authoritative AUT leg. This closes the false positive where a driver-side call alone could pass even if the plugin placed no compliant outbound call.
  • Updated — tests/live/test_cross_channel.py::test_sms_request_gets_call: adds the same paired-leg and persisted voicemail verification for SMS-triggered calls. This closes stale-call and wrong-owner matches.
  • Updated — tests/live/test_sms.py::test_sms_retry_after_carrier_delivery_failure: increases bounded conversation-history inspection so the test can still locate its current correlated record on busy shared live identities. This closes a history-window false negative without relaxing recipient, correlation, retry, or delivery assertions.
  • Updated — tests/live/test_email_intelligence.py::test_reports_sender_details: requires the requesting agent to report the sender's complete email address and phone number, preserving the existing API-side assertions while removing ambiguity in the natural-language task.
  • Updated — live CI orchestration: gives the channels job a 25-minute ceiling, retains the voice job's existing 15-minute ceiling, bounds gateway-readiness curl attempts, and prints cancellation diagnostics when a matrix child prevents the aggregate gate from settling.

Exact-head CI

  • PR checks run 30700714915: passed on 91c20335386e86e1c665a76cc1b5c05803e493b2 (Python 3.11, Python 3.12, and current-host contract).
  • Full-stack run 30700714990: passed on the same exact head, including mock and real channels, all four A2A variants, inbound TTS/STT, hosted Voice AI, outbound Realtime, Hermes' direct-contact Realtime path, external events, and the aggregate gate.
  • No deployment, package publication, image build, or merge was performed from this repository.

Related PRs

@dimavrem22 dimavrem22 changed the title docs: plan phone call voice stack support feat: add phone call voice stack support Jul 29, 2026
@dimavrem22
dimavrem22 marked this pull request as draft July 29, 2026 23:45
@dimavrem22
dimavrem22 force-pushed the plan/phone-call-voice-stack branch from 03d29d1 to 6358226 Compare July 30, 2026 09:37
@dimavrem22 dimavrem22 changed the title feat: add phone call voice stack support feat: add selectable phone call voice stacks Jul 30, 2026
@dimavrem22
dimavrem22 force-pushed the plan/phone-call-voice-stack branch from 90fd2c8 to 153ae6a Compare July 31, 2026 19:22
@dimavrem22
dimavrem22 marked this pull request as ready for review July 31, 2026 21:30
@dimavrem22
dimavrem22 merged commit 7d7d8fb into main Aug 1, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant