Feat/fastino privacy mode

.env.example

@@ -1,17 +1,30 @@
 REDIS_URL=redis://redis:6379/0
 CELERY_CONCURRENCY=1
 NEXT_PUBLIC_API_BASE_URL=http://localhost:8000
+NEXT_PUBLIC_VOICE_SERVER_URL=http://localhost:7860
+NEXT_PUBLIC_PATENTFLOW_API_KEY=
 ALLOWED_ORIGINS=http://localhost:3000,http://localhost:3001
 STATUS_RESPONSE_MAX_BYTES=262144
 RESULT_BACKEND_MAX_BYTES=262144
+PATENTFLOW_API_KEY=
 
 EPO_ENABLED=false
 EPO_CONSUMER_KEY=
 EPO_CONSUMER_SECRET=
 
 LLM_PROVIDER=local
-LLM_BASE_URL=http://127.0.0.1:11434
+LLM_BASE_URL=http://host.docker.internal:11434/v1
 LLM_MODEL=
+LLM_API_KEY=
+
+# Fastino Pioneer fine-tuned PatentFlow attorney model
+ENABLE_LOCAL_PRIVACY_MODE=false
+FASTINO_API_KEY=
+FASTINO_BASE_URL=https://api.pioneer.ai/v1
+FASTINO_INFERENCE_ENDPOINT=https://api.pioneer.ai/v1/inference
+FASTINO_MODEL_NAME=PatentFlow-epo-patent-attorney-llama-3-1-8b
+FASTINO_MODEL_ID=59d36fbf-6e40-4e07-96d5-617d321842e8
+
 RAG_EMBEDDING_BASE_URL=
 RAG_EMBEDDING_MODEL=
 

.gitignore

@@ -18,6 +18,9 @@ dist/
 *.egg-info/
 
 .DS_Store
+Icon?
+**/Icon?
+.voice-venv/
 
 .vscode/
 .idea/

README.md

@@ -25,10 +25,10 @@ Patent prosecution is not “generic writing.” It is **risk management**:
 - Inventive-step reasoning requires structured, repeatable mapping
 - Quality and traceability matter more than “chatty” UX
 
-### 2) 100% offline operation for client confidentiality
-PatentFlow is designed to run fully locally:
-- Local LLM execution (air-gapped capable)
-- No external SaaS dependencies required for core workflows
+### 2) Privacy-aware hybrid operation for client confidentiality
+PatentFlow supports both local and online inference paths:
+- Sensitive files and privacy-mode requests can be routed to Local LLM or a controlled Pioneer by Fastino endpoint
+- Non-sensitive tasks can use configured online OpenAI-compatible APIs such as OpenAI, OpenRouter, r9s.ai, or vLLM-compatible services
 - Local persistence for attorney-specific preferences
 
 ### 3) Enterprise UX: minimal, information-dense, institutional
@@ -55,6 +55,12 @@ graph TD
         UI -->|POST /api/generate| API[FastAPI Gateway :8000]
         UI -->|GET /api/status/:id| API
         UI -->|GET/POST /api/memory/*| API
+        UI -->|Voice Controls| VoiceS[Voice Server :7860]
+    end
+
+    subgraph Voice [Gradbot Voice Pipeline - Port 7860]
+        VoiceS -->|STT → LLM → TTS| GradbotRT[Gradbot Runtime]
+        GradbotRT -->|LLM Calls| API
     end
 
     subgraph Backend [FastAPI + Celery Workers]
@@ -70,8 +76,11 @@ graph TD
         EPO[EPO API] -->|Prior Art Retrieval| API
     end
 
-    subgraph AI [Local AI Runtime]
-        Skills --> LLM[(Local LLM)]
+    subgraph AI [AI Runtime & Model Routing]
+        Skills --> Router[Privacy / Sensitivity Router]
+        Router -->|Sensitive files or Privacy Mode| LocalLLM[(Local LLM)]
+        Router -->|Non-sensitive tasks| OnlineAPI[(Online LLM API<br/>OpenAI-compatible)]
+        Router -->|Optional trained route| Fastino[(Pioneer by Fastino<br/>PatentFlow Attorney Model)]
     end
 ```
 
@@ -116,6 +125,45 @@ PatentFlow integrates EPO retrieval to support:
 - Increases completeness and consistency of cited-document context
 - Improves auditability of the evidence basis used in analysis
 
+### 5) Real-Time Voice Interaction (Gradbot)
+PatentFlow includes a real-time voice session for attorney-in-the-loop workflows:
+- **Start the session** opens a browser-based voice channel to the PatentFlow engine
+- Discuss claim charts, examiner objections, and draft strategy conversationally
+- Voice pipeline uses Gradbot for streaming STT → LLM → TTS with sub-second latency
+- Session state is ephemeral; no audio stored or logged
+
+**Business value**
+- Reduces typing friction during document review
+- Enables rapid "talk through" of objection strategy before committing to text
+- Maintains client confidentiality: voice processing runs locally, no cloud transcription
+
+### 6) Privacy Mode (Pioneer by Fastino)
+PatentFlow includes an opt-in inference path powered by our own Pioneer by Fastino trained model for EPO prosecution work.
+
+**Trained model**
+- Platform: Pioneer by Fastino
+- Model Name: `PatentFlow-epo-patent-attorney-llama-3-1-8b`
+- Job ID / Model ID: `59d36fbf-6e40-4e07-96d5-617d321842e8`
+- Base URL: `https://api.pioneer.ai/v1`
+- Inference Endpoint: `https://api.pioneer.ai/v1/inference`
+- Training dataset: the curated fine-tuning dataset is included in the repository root as `dataset for training model for PatentFlow Attorney.jsonl`
+
+**Why this model**
+- EPO-specific: trained for European patent attorney workflows, including Art. 56 inventive-step reasoning and Art. 123(2) added-matter risk review
+- Structured outputs: optimized for claim charts, objection extraction, severity labels, examiner reasoning, and recommended attorney actions
+- Domain tone: produces prosecution-oriented analysis instead of generic legal summaries
+- Privacy-aware routing: lets PatentFlow keep the default workflow unchanged while offering a dedicated controlled endpoint for sensitive patent analysis
+
+**How it is used**
+- Toggle globally with `ENABLE_LOCAL_PRIVACY_MODE=true` in `.env`
+- Or toggle per request from the frontend with `Privacy Mode (Local LLM)`
+- When disabled, the existing LLM workflow operates unchanged
+- When enabled, Pioneer by Fastino is tried first; timeout, invalid JSON, or API failure falls back gracefully
+
+**Business value**
+- Adds patent-prosecution-specific model behavior without forking the pipeline
+- Preserves the existing public/online workflow by default
+
 ---
 
 ## Quick Start
@@ -124,6 +172,7 @@ PatentFlow integrates EPO retrieval to support:
 1) Configure environment:
 - Copy `.env.example` → `.env`
 - Set `NEXT_PUBLIC_API_BASE_URL`, `REDIS_URL`, and LLM configuration as needed
+- For Fastino privacy mode, set `FASTINO_API_KEY`, `FASTINO_BASE_URL`, and `FASTINO_MODEL_ID`; leave `ENABLE_LOCAL_PRIVACY_MODE=false` unless you want Fastino as the default route
 
 2) Start services:
 ```bash
@@ -140,8 +189,8 @@ Typical services:
 
 #### 1) Backend (FastAPI)
 ```bash
-python3 -m venv venv
-source venv/bin/activate
+python3 -m venv .venv
+source .venv/bin/activate
 pip install -r requirements.txt
 
 export REDIS_URL=redis://localhost:6379/0
@@ -166,6 +215,14 @@ npm install
 npm run dev
 ```
 
+#### 5) Voice Server (Gradbot)
+```bash
+python3 -m venv .voice-venv
+source .voice-venv/bin/activate
+pip install -r voice_pipeline/requirements.txt
+python -m voice_pipeline.server
+```
+
 Open:
 - UI: `http://localhost:3000`
 - API: `http://localhost:8000/health`
@@ -175,6 +232,7 @@ Open:
 ## API Overview (Selected)
 - `POST /api/generate`
   - Runs the async pipeline (Celery) for claim chart + verification + draft outputs
+  - Accepts optional `use_privacy_mode: true` to try Fastino Pioneer for this request
 - `GET /api/status/{task_id}`
   - Poll for progress and results
 - `GET /api/memory/{attorney_id}`
@@ -184,17 +242,53 @@ Open:
 - `POST /api/generate-chart`
   - Deterministic + LLM-assisted chart generation with optional `attorney_id`
 
+### Voice Session Endpoints
+- `POST /start_bot`
+  - Initialize a voice session with attorney context prompt; returns `session_id` + WebSocket endpoint
+- `POST /end_session/{session_id}`
+  - Terminate voice session and release server-side resources
+- `WS /ws/chat`
+  - Bidirectional streaming channel for voice interaction
+
+---
+
+## Pioneer by Fastino Configuration
+Add these values to `.env` when using Privacy Mode:
+
+```env
+ENABLE_LOCAL_PRIVACY_MODE=false
+FASTINO_API_KEY=
+FASTINO_BASE_URL=https://api.pioneer.ai/v1
+FASTINO_INFERENCE_ENDPOINT=https://api.pioneer.ai/v1/inference
+FASTINO_MODEL_NAME=PatentFlow-epo-patent-attorney-llama-3-1-8b
+FASTINO_MODEL_ID=59d36fbf-6e40-4e07-96d5-617d321842e8
+```
+
+Notes:
+- `ENABLE_LOCAL_PRIVACY_MODE=false` keeps the existing route as default.
+- The frontend toggle sends `use_privacy_mode: true` for a single request.
+- Do not commit `FASTINO_API_KEY` or local `.env`.
+
 ---
 
 ## Security & Privacy Posture
-- Designed for offline and air-gapped operation
+- Privacy-aware routing: sensitive files and privacy-mode requests can be routed away from general online APIs
 - Local persistence only (SQLite)
-- No dependency on third-party analytics, telemetry, or cloud inference for core workflows
+- No dependency on third-party analytics or telemetry
+- Online inference is explicit and configurable for non-sensitive tasks through OpenAI-compatible APIs
+- Voice sessions are ephemeral: no audio retention, transcription, or logging
+- Pioneer by Fastino privacy mode routes supported analysis through the configured Pioneer endpoint with fallback to the existing route
 
 ---
 
 ## Roadmap (Prioritized for Firm Integration)
 
+- [ ] **Two-Tier AI Architecture (Performance & UX)**  
+  Implement a dual-layer local inference pipeline using the Fastino platform to optimize both speed and compute efficiency for privacy-mode:
+  - **Layer 1 (Zero-Shot Triage):** Utilize GLiNER2 for ultra-fast, zero-shot entity extraction. Instantly highlight critical legal entities (e.g., Prior Art documents like D1/D2, disputed technical features, and statutory articles) directly in the UI milliseconds after document upload.
+  - **Layer 2 (Deep Legal Reasoning):** While the attorney engages with the instantly highlighted document, the fine-tuned local Llama-3.1-8B model asynchronously processes the complex Art. 56 / Art. 123(2) logic in the background via Celery.
+  This architecture provides immediate visual feedback to the attorney without blocking the UI or overloading the heavy LLM.
+
 - [ ] **RAG with ChromaDB (Depth)**  
   Local retrieval over firm-approved corpora (e.g., standards, prior OA templates) to improve long-document reasoning while controlling hallucination risk.
 

dataset for training model for PatentFlow Attorney.jsonl

@@ -0,0 +1,4 @@
+{"messages":[{"role":"system","content":"You are a European Patent Attorney assistant. Extract EPC objections from the Office Action text and output ONLY valid JSON matching the specified schema."},{"role":"user","content":"Extract EPC objections: 'Regarding Claim 1, the term \"high temperature\" is vague and leaves the reader in doubt as to the meaning of the technical features to which it refers, thereby rendering the definition of the subject-matter of said claim unclear (Article 84 EPC). The applicant should specify the exact temperature range in degrees Celsius.'"},{"role":"assistant","content":"{\n  \"article\": \"84\",\n  \"affected_claims\": [1],\n  \"severity\": \"high\",\n  \"examiner_reasoning\": \"The term 'high temperature' is vague and does not define a clear boundary for the scope of protection.\",\n  \"recommended_action\": \"Amend claim 1 to replace 'high temperature' with a specific temperature range in degrees Celsius, ensuring support in the original description.\"\n}"}]}
+{"messages":[{"role":"system","content":"You are a European Patent Attorney assistant. Extract EPC objections from the Office Action text and output ONLY valid JSON matching the specified schema."},{"role":"user","content":"Extract EPC objections: 'The amendment to Claim 3 introduces the feature \"a casing made of titanium alloy\". However, the original description on page 5, line 12 only generically discloses a \"metal casing\". There is no basis in the application as filed for selecting titanium alloy. This constitutes added subject-matter extending beyond the content of the application as filed, contrary to Article 123(2) EPC.'"},{"role":"assistant","content":"{\n  \"article\": \"123(2)\",\n  \"affected_claims\": [3],\n  \"severity\": \"high\",\n  \"examiner_reasoning\": \"The specific material 'titanium alloy' was not disclosed in the original application, which only mentioned 'metal casing'.\",\n  \"recommended_action\": \"Delete the feature 'made of titanium alloy' from Claim 3, or revert to the original broader term 'metal casing', unless a valid basis can be found elsewhere in the originally filed drawings or description.\"\n}"}]}
+{"messages":[{"role":"system","content":"You are a European Patent Attorney assistant. Extract EPC objections from the Office Action text and output ONLY valid JSON matching the specified schema."},{"role":"user","content":"Extract EPC objections: 'The present application does not meet the requirements of Article 52(1) in conjunction with Article 56 EPC because the subject-matter of claim 1 does not involve an inventive step. Document D1 (US 2018/01234) is regarded as being the closest prior art and discloses a wireless charging pad. The subject-matter of claim 1 differs from D1 in that it includes an active cooling fan. The objective technical problem to be solved is how to prevent overheating. However, D2 (EP 312456) already teaches using a cooling fan in inductive chargers. It would be obvious to the skilled person to apply the teaching of D2 to D1.'"},{"role":"assistant","content":"{\n  \"article\": \"56\",\n  \"affected_claims\": [1],\n  \"severity\": \"high\",\n  \"examiner_reasoning\": \"D1 discloses the closest prior art. The distinguishing feature is an active cooling fan solving the problem of overheating. D2 renders this combination obvious to a person skilled in the art.\",\n  \"recommended_action\": \"Argue why the combination of D1 and D2 would not be obvious (e.g., teaching away, incompatible structures), or amend claim 1 to include a further distinguishing feature not found in D1 or D2.\"\n}"}]}
+{"messages":[{"role":"system","content":"You are a European Patent Attorney assistant. Extract EPC objections from the Office Action text and output ONLY valid JSON matching the specified schema."},{"role":"user","content":"Extract EPC objections: 'We are pleased to inform you that the renewal fee for the 4th year has been successfully processed and the receipt is attached herewith.'"},{"role":"assistant","content":"{\n  \"article\": \"None\",\n  \"affected_claims\": [],\n  \"severity\": \"low\",\n  \"examiner_reasoning\": \"The provided text is an administrative notification regarding fee payment, not a substantive examination report.\",\n  \"recommended_action\": \"No substantive response required. Keep the receipt for records.\"\n}"}]}

docker-compose.yml

@@ -19,6 +19,8 @@ services:
     build:
       context: .
       dockerfile: Dockerfile.api
+    env_file:
+      - .env
     environment:
       - REDIS_URL=redis://redis:6379/0
     networks:
@@ -47,6 +49,8 @@ services:
     build:
       context: .
       dockerfile: Dockerfile.worker
+    env_file:
+      - .env
     environment:
       - REDIS_URL=redis://redis:6379/0
       - CELERY_CONCURRENCY=1
@@ -63,7 +67,9 @@ services:
       context: ./frontend
       dockerfile: Dockerfile
       args:
-        NEXT_PUBLIC_API_BASE_URL: http://localhost:8000
+        NEXT_PUBLIC_API_BASE_URL: ${NEXT_PUBLIC_API_BASE_URL:-http://localhost:8000}
+        NEXT_PUBLIC_PATENTFLOW_API_KEY: ${NEXT_PUBLIC_PATENTFLOW_API_KEY:-}
+        NEXT_PUBLIC_VOICE_SERVER_URL: ${NEXT_PUBLIC_VOICE_SERVER_URL:-http://localhost:7860}
     networks:
       - patentflow-net
     ports:

frontend/.dockerignore

@@ -0,0 +1,13 @@
+node_modules/
+.next/
+out/
+.git/
+.gitignore
+.DS_Store
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+.env
+.env.local
+.env.*.local

frontend/Dockerfile

@@ -12,6 +12,10 @@ COPY . /app
 # NEXT_PUBLIC_* vars are baked at build time
 ARG NEXT_PUBLIC_API_BASE_URL=http://localhost:8000
 ENV NEXT_PUBLIC_API_BASE_URL=${NEXT_PUBLIC_API_BASE_URL}
+ARG NEXT_PUBLIC_PATENTFLOW_API_KEY=
+ENV NEXT_PUBLIC_PATENTFLOW_API_KEY=${NEXT_PUBLIC_PATENTFLOW_API_KEY}
+ARG NEXT_PUBLIC_VOICE_SERVER_URL=http://localhost:7860
+ENV NEXT_PUBLIC_VOICE_SERVER_URL=${NEXT_PUBLIC_VOICE_SERVER_URL}
 
 RUN npm run build