Knowledge-Enhanced Chat Integration

GitHub Issue: #249 Date: 2025-09-29 Status: Planning Priority: High - Core AutoBot functionality

Overview

Integrate the Knowledge Manager's 14,000+ vector-indexed facts into the chat system to enable:

Contextual responses using stored knowledge
RAG (Retrieval-Augmented Generation) for accurate answers
Automatic knowledge retrieval based on conversation context
Source citations from knowledge base

Current Architecture

Knowledge Manager (Operational)

Location: src/knowledge_base_v2.py
Vectors: 14,047 indexed with 768-dimensional embeddings
Search: Semantic search via LlamaIndex + Redis
API: /api/knowledge_base/search endpoint available

Chat System

Location: autobot-backend/api/chat.py
LLM: Ollama integration (6-tier routing; chat uses Quality tier: qwen3.5:9b)
WebSocket: Real-time streaming responses
State: Manages conversation history

Recommended Integration Approach

Option 1: RAG Pipeline Integration (RECOMMENDED)

Architecture:

User Query -> Query Analysis -> Knowledge Retrieval -> Context Augmentation -> LLM Response

Implementation:

Query Analysis Layer
- Detect if query benefits from knowledge base
- Extract key terms for semantic search
- Determine search scope (category, recency, etc.)

Knowledge Retrieval

# In chat endpoint before LLM call
relevant_facts = await knowledge_base.search(
    query=user_message,
    top_k=5  # Retrieve top 5 most relevant facts
)

Context Augmentation

# Build enhanced prompt with retrieved knowledge
context = "\n\n".join([f"- {fact['content']}" for fact in relevant_facts])

augmented_prompt = f"""You are AutoBot assistant. Answer using the following knowledge context:

KNOWLEDGE CONTEXT:
{context}

USER QUESTION: {user_message}

Provide an accurate answer based on the knowledge context. If the context doesn't contain relevant information, say so."""

LLM Generation
- Send augmented prompt to Ollama
- Stream response back to user
- Include source citations

File Changes:

autobot-backend/api/chat.py: Add RAG pipeline before LLM call
autobot-backend/agents/chat_agent.py: Integrate knowledge retrieval
autobot-backend/agents/rag_agent.py: Enhanced RAG orchestration

Advantages:

Most accurate - uses actual stored knowledge
Cites sources from knowledge base
Reduces hallucinations
Handles 14,000+ facts efficiently
Automatic relevance ranking via vector search

Considerations:

Latency: +100-300ms for knowledge retrieval
Context window: Must fit retrieved facts + query + response
Relevance threshold: Only use highly relevant facts (score > 0.7)

Option 2: Agent-Based Knowledge Integration

Architecture:

User Query -> Agent Router -> Knowledge Agent (if needed) -> Response Agent -> User

Implementation:

Create Knowledge-Aware Agent

# autobot-backend/agents/knowledge_chat_agent.py
class KnowledgeChatAgent:
    async def should_use_knowledge(self, query: str) -> bool:
        """Determine if query needs knowledge base"""
        keywords = ["what is", "how to", "explain", "tell me about"]
        return any(kw in query.lower() for kw in keywords)

    async def get_response(self, query: str) -> str:
        if await self.should_use_knowledge(query):
            facts = await self.kb.search(query, top_k=3)
            return await self.generate_with_knowledge(query, facts)
        else:
            return await self.generate_direct(query)

Agent Orchestration
- Route queries through agent system
- Knowledge agent decides when to retrieve
- Other agents handle non-knowledge queries

File Changes:

autobot-backend/agents/knowledge_chat_agent.py: New agent
autobot-backend/agents/agent_orchestrator.py: Route to knowledge agent
autobot-backend/api/chat.py: Use orchestrator

Advantages:

Flexible - can combine multiple agents
Extensible - easy to add more agents
Selective - only uses KB when beneficial

Considerations:

More complex architecture
Requires agent routing logic
Potential for routing errors

Option 3: Hybrid Approach (BEST FOR AUTOBOT)

Combine RAG + Agents for optimal flexibility:

# autobot-backend/api/chat.py - Enhanced chat endpoint

async def chat_with_knowledge(message: str, session_id: str):
    # 1. Check if knowledge might be helpful
    query_analysis = await analyze_query_intent(message)

    # 2. Retrieve knowledge if beneficial
    knowledge_context = []
    if query_analysis.needs_knowledge:
        knowledge_context = await knowledge_base.search(
            query=message,
            top_k=5
        )

    # 3. Route through agent orchestrator with knowledge
    response = await agent_orchestrator.process(
        message=message,
        knowledge_context=knowledge_context,
        session_id=session_id
    )

    # 4. Stream response with citations
    async for chunk in response:
        yield chunk

Advantages:

Best of both worlds
Efficient knowledge retrieval
Agent flexibility
Automatic relevance detection

Implementation Plan

Phase 1: Basic RAG Integration (2-4 hours)

Add RAG to Chat Endpoint

# File: autobot-backend/api/chat.py

@router.post("/message")
async def chat_message(request: ChatRequest):
    # Get knowledge base instance
    kb = request.app.state.knowledge_base

    # Search for relevant knowledge
    relevant_facts = await kb.search(request.message, top_k=5)

    # Build context
    if relevant_facts:
        context = "\n".join([f"- {f['content']}" for f in relevant_facts])
        augmented_message = f"Context: {context}\n\nQuestion: {request.message}"
    else:
        augmented_message = request.message

    # Get LLM response with context
    response = await llm_interface.chat(augmented_message)
    return response

Add Knowledge Toggle
- Frontend checkbox: "Use knowledge base"
- API parameter: use_knowledge: bool = True
- Allow users to disable for pure LLM chat
Add Citation Display
- Return source fact IDs with response
- Display "Sources: [1] [2] [3]" in UI
- Allow clicking sources to view full fact

Phase 2: Smart Knowledge Retrieval (4-6 hours)

Query Intent Detection

async def analyze_query_intent(query: str) -> QueryAnalysis:
    """Determine if query needs knowledge"""
    # Patterns that benefit from knowledge
    knowledge_patterns = [
        r"what is",
        r"how (do|to)",
        r"explain",
        r"tell me about",
        r"define",
        r"describe"
    ]

    needs_knowledge = any(
        re.search(pattern, query.lower())
        for pattern in knowledge_patterns
    )

    return QueryAnalysis(
        needs_knowledge=needs_knowledge,
        confidence=0.8 if needs_knowledge else 0.2
    )

Relevance Filtering
- Only use facts with score > 0.7
- Limit to top 3-5 most relevant
- Avoid irrelevant context pollution
Category-Aware Search
- Extract category hints from query
- Filter knowledge by category
- Improve search precision

Phase 3: Advanced Features (6-8 hours)

Conversation-Aware RAG
- Track conversation topics
- Maintain relevant knowledge cache
- Update search based on conversation flow
Multi-turn Knowledge
- Remember what knowledge was used
- Avoid re-retrieving same facts
- Build conversation knowledge graph
Knowledge Updates from Chat
- "Remember this: " command
- Store new facts during conversation
- Automatic fact extraction

API Endpoints

Enhanced Chat Endpoint

POST /api/chat/message
Content-Type: application/json

{
  "message": "What is the ls command?",
  "session_id": "abc123",
  "use_knowledge": true,
  "knowledge_options": {
    "top_k": 5,
    "score_threshold": 0.3,
    "categories": ["commands"]
  }
}

Response (streaming):
{
  "content": "The ls command lists directory contents...",
  "sources": [
    {
      "fact_id": "uuid-123",
      "content": "ls - list directory contents",
      "score": 0.95,
      "category": "commands"
    }
  ],
  "used_knowledge": true
}

Knowledge Search Endpoint (Existing)

POST /api/knowledge_base/search
Content-Type: application/json

{
  "query": "ls command",
  "top_k": 5
}

Response:
[
  {
    "content": "ls - list directory contents",
    "score": 0.95,
    "metadata": {"category": "commands"},
    "node_id": "uuid-123"
  }
]

Frontend Integration

Chat Interface Updates

Knowledge Toggle

<template>
  <div class="chat-controls">
    <label>
      <input type="checkbox" v-model="useKnowledge" />
      Use Knowledge Base
    </label>
  </div>
</template>

Source Citations

<template>
  <div class="message-sources" v-if="message.sources">
    <span>Sources:</span>
    <a
      v-for="source in message.sources"
      :key="source.fact_id"
      @click="viewSource(source)"
      class="source-link"
    >
      [{{ source.score.toFixed(2) }}]
    </a>
  </div>
</template>

Knowledge Indicator
- Show icon when knowledge is used
- Display "Using knowledge base..." during retrieval
- Show number of facts retrieved

Knowledge Panel

Retrieved Facts Display
- Show facts used in current response
- Highlight relevant portions
- Allow expanding to full fact
Knowledge Search
- Standalone search interface
- Browse knowledge by category
- View all indexed facts

Performance Considerations

Latency Targets

Knowledge retrieval: < 200ms
Context augmentation: < 50ms
Total overhead: < 250ms
Streaming start: < 500ms (total)

Optimization Strategies

Caching

@lru_cache(maxsize=100)
async def get_cached_knowledge(query_hash: str):
    return await knowledge_base.search(query)

Parallel Processing

# Retrieve knowledge and start LLM simultaneously
knowledge_task = asyncio.create_task(kb.search(query))
llm_task = asyncio.create_task(llm.prepare(query))

knowledge = await knowledge_task
# Augment and continue with LLM

Relevance Pre-filtering
- Quick keyword check before vector search
- Skip retrieval for obviously irrelevant queries
- Reduce unnecessary vector operations

Testing Strategy

Unit Tests

Knowledge Retrieval

async def test_knowledge_retrieval():
    facts = await kb.search("ls command", top_k=5)
    assert len(facts) > 0
    assert facts[0]['score'] > 0.7

Context Augmentation

async def test_context_building():
    facts = [{"content": "ls lists files"}]
    context = build_context(facts)
    assert "ls lists files" in context

RAG Pipeline

async def test_rag_pipeline():
    response = await chat_with_knowledge("what is ls?")
    assert "lists" in response.lower()
    assert len(response.sources) > 0

Integration Tests

End-to-End Chat
- Send query through API
- Verify knowledge retrieval
- Check response quality
- Validate source citations
Performance Tests
- Measure retrieval latency
- Test with 1000+ concurrent queries
- Verify caching effectiveness

Monitoring & Metrics

Key Metrics to Track

Usage Metrics
- % of queries using knowledge
- Average facts retrieved per query
- Knowledge vs non-knowledge queries
Performance Metrics
- Knowledge retrieval time (p50, p95, p99)
- Total RAG overhead
- Cache hit rate
Quality Metrics
- Average relevance score
- User satisfaction (thumbs up/down)
- Knowledge usefulness rate

Logging

logger.info(
    "Knowledge retrieval",
    extra={
        "query": query,
        "facts_retrieved": len(facts),
        "top_score": facts[0]['score'] if facts else 0,
        "retrieval_ms": retrieval_time,
        "used_in_response": True
    }
)

Security & Privacy

Knowledge Access Control
- Filter knowledge by user permissions
- Respect category access levels
- Audit knowledge access
Sensitive Information
- Mark facts as sensitive/private
- Exclude from general retrieval
- Require explicit access
Rate Limiting
- Limit knowledge queries per minute
- Prevent knowledge base DoS
- Cache to reduce load

Next Steps

Immediate (Week 1)

Fix stats display (COMPLETED)
Implement basic RAG integration
Add knowledge toggle to UI
Test with sample queries

Short-term (Week 2-3)

Add smart query intent detection
Implement relevance filtering
Add source citations display
Performance optimization

Long-term (Month 2)

Conversation-aware RAG
Multi-turn knowledge tracking
Knowledge updates from chat
Advanced analytics

Navigation Menu

Repository navigation

Knowledge-Enhanced Chat Integration

Overview

Current Architecture

Knowledge Manager (Operational)

Chat System

Recommended Integration Approach

Option 1: RAG Pipeline Integration (RECOMMENDED)

Option 2: Agent-Based Knowledge Integration

Option 3: Hybrid Approach (BEST FOR AUTOBOT)

Implementation Plan

Phase 1: Basic RAG Integration (2-4 hours)

Phase 2: Smart Knowledge Retrieval (4-6 hours)

Phase 3: Advanced Features (6-8 hours)

API Endpoints

Enhanced Chat Endpoint

Knowledge Search Endpoint (Existing)

Frontend Integration

Chat Interface Updates

Knowledge Panel

Performance Considerations

Latency Targets

Optimization Strategies

Testing Strategy

Unit Tests

Integration Tests

Monitoring & Metrics

Key Metrics to Track

Logging

Security & Privacy

Next Steps

Immediate (Week 1)

Short-term (Week 2-3)

Long-term (Month 2)

Related Documentation

FilesExpand file tree

knowledge_chat_integration.md

Latest commit

History

knowledge_chat_integration.md

File metadata and controls

Knowledge-Enhanced Chat Integration

Overview

Current Architecture

Knowledge Manager (Operational)

Chat System

Recommended Integration Approach

Option 1: RAG Pipeline Integration (RECOMMENDED)

Option 2: Agent-Based Knowledge Integration

Option 3: Hybrid Approach (BEST FOR AUTOBOT)

Implementation Plan

Phase 1: Basic RAG Integration (2-4 hours)

Phase 2: Smart Knowledge Retrieval (4-6 hours)

Phase 3: Advanced Features (6-8 hours)

API Endpoints

Enhanced Chat Endpoint

Knowledge Search Endpoint (Existing)

Frontend Integration

Chat Interface Updates

Knowledge Panel

Performance Considerations

Latency Targets

Optimization Strategies

Testing Strategy

Unit Tests

Integration Tests

Monitoring & Metrics

Key Metrics to Track

Logging

Security & Privacy

Next Steps

Immediate (Week 1)

Short-term (Week 2-3)

Long-term (Month 2)

Related Documentation