🩺 MedAI - Intelligent Pre-diagnosis Platform

🏆 Built for the Google Healthcare Hackathon by EPITA Students

An automated pre-diagnosis tool and patient scheduling prioritization system powered by MedGemma AI

🌟 Project Overview

MedAI is an innovative healthcare application that combines AI-powered pre-diagnosis with realistic virtual consultations. The platform uses Google's MedGemma AI model and HeyGen's streaming avatar technology to provide interactive medical consultations and intelligent patient prioritization.

🎯 Key Features

• 🤖 AI Virtual Doctor: Interactive consultations with HeyGen streaming avatars
• 📊 Smart Prioritization: Automated patient scheduling based on urgency assessment
• 📋 Medical Reports: Comprehensive patient dashboard with PDF report viewing
• 📈 Patient Timeline: Historical tracking of medical consultations and reports
• 🎨 Modern UI: Beautiful, responsive interface built with React and Tailwind CSS

🚀 Live Demo

• 🏠 Homepage: Access the main platform interface
• 📞 Virtual Consultation: /call/:id - Interactive AI consultation with HeyGen avatar
• 📊 Doctor Dashboard: /report/:doctorID - Patient management interface
• 📄 Report Details: /report/:doctorID/:reportID - Detailed patient reports with PDF viewing

🏗️ Project Structure

user-interface/
├── 📁 src/
│   ├── 🎭 animations/           # Framer Motion animations
│   │   └── callAnimations.ts    # Call interface animations
│   ├── 🔌 api/                  # API integrations
│   │   └── heygen-token/        # HeyGen authentication
│   │       └── getAccessToken.ts
│   ├── 🧩 components/           # React components
│   │   ├── callComponents/      # Video call interface
│   │   │   ├── CallVideo.tsx    # HeyGen avatar video component
│   │   │   ├── CallNavbar.tsx   # Call controls
│   │   │   ├── CallSidebar.tsx  # Chat sidebar
│   │   │   └── ...
│   │   ├── mainComponents/      # Shared components
│   │   │   ├── AnimatedBackground.tsx
│   │   │   └── Footer.tsx
│   │   └── reportComponents/    # Medical reports interface
│   │       ├── DoctorNavbar.tsx
│   │       ├── PatientRow.tsx
│   │       ├── PatientTimeline.tsx
│   │       └── ...
│   ├── 🎨 constants/            # App constants
│   │   └── colors.ts            # Theme colors
│   ├── 🤖 heygen/               # HeyGen integration
│   │   └── StreamingAvatarContext.tsx
│   ├── 📄 pages/                # Route components
│   │   ├── Home.tsx             # Landing page
│   │   ├── Call.tsx             # Virtual consultation
│   │   ├── DoctorReports.tsx    # Patient dashboard
│   │   ├── ReportDetail.tsx     # Individual reports
│   │   └── NotFound.tsx         # 404 page
│   └── ...
├── 📁 public/                   # Static assets
│   ├── team/                    # Team member photos
│   └── sample-report.pdf        # Sample medical report
├── 📦 package.json              # Dependencies & scripts
├── ⚙️ vite.config.ts           # Vite configuration
├── 🎨 tailwind.config.js       # Tailwind CSS config
└── 📖 README.md                # This file

🤖 HeyGen Integration Deep Dive

🔧 Core Implementation

The HeyGen streaming avatar integration is the heart of our virtual consultation system:

StreamingAvatarContext (src/heygen/StreamingAvatarContext.tsx)

// Key features implemented:
- Real-time video streaming with WebRTC
- Voice emotion and speech synthesis
- Speech-to-text integration with Deepgram
- Avatar state management (connecting, ready, speaking)
- Session lifecycle management

Avatar States

• DISCONNECTED - Initial state
• CONNECTING - Establishing connection
• CONNECTED - Connected but not ready
• READY - Ready for interaction
• SPEAKING - Avatar is currently speaking
• ERROR - Connection error

Key Features

• 🎙️ Voice Chat: WebSocket-based real-time communication
• 🎭 Avatar: "Ann_Doctor_Sitting_public" medical avatar
• 🗣️ TTS: Text-to-speech with emotional voice settings
• 👂 STT: Speech-to-text using Deepgram provider
• 📹 High Quality: HD video streaming with optimized performance

🔐 Authentication Flow

// Token-based authentication
const HEYGEN_API_KEY = import.meta.env.VITE_HEYGEN_TOKEN;

// Secure token retrieval
export async function get_access_token() {
  const res = await fetch(`https://api.heygen.com/v1/streaming.create_token`, {
    method: "POST",
    headers: { "x-api-key": HEYGEN_API_KEY },
  });
  return res.json().data.token;
}

🎯 Usage in Components

// In Call component
const { speakText, isReady, mediaStream, status } = useStreamingAvatar();

// Send message to avatar
await speakText("Hello, how can I help you today?");

// Video rendering
<video ref={videoRef} autoPlay muted playsInline />;

🛠️ Technology Stack

Frontend Framework

• - Latest React with concurrent features
• - Type-safe development
• - Fast build tooling

Styling & Animation

• - Utility-first CSS
• - Advanced animations

AI & Media Processing

• - Streaming avatar technology
• - PDF document viewing
• - PDF rendering engine

State Management & Routing

• - Lightweight state management
• - Client-side routing

🚀 Getting Started

Prerequisites

• Node.js (v18 or higher)
• npm or yarn
• HeyGen API Key (for avatar functionality)

Installation

1. Clone the repository

   git clone <repository-url>
   cd user-interface

2. Install dependencies

   npm install

3. Environment Setup

   # Create .env file
   echo "VITE_HEYGEN_TOKEN=your_heygen_api_key_here" > .env

4. Start development server

   npm run dev

5. Open in browser

   http://localhost:5173
   

🔧 Available Scripts

npm run dev      # Start development server
npm run build    # Build for production
npm run preview  # Preview production build
npm run lint     # Run ESLint

🎨 UI/UX Features

Design Principles

• 🏥 Medical-First: Clean, professional healthcare interface
• ♿ Accessible: WCAG compliant design patterns
• 📱 Responsive: Mobile-first responsive design
• 🎭 Animated: Smooth Framer Motion animations
• 🌟 Modern: Glass morphism and modern UI patterns

Color Scheme

/* Primary Healthcare Colors */
--color-primary: #2563eb    /* Professional blue */
--color-accent: #60a5fa     /* Light blue accent */
--color-success: #10b981    /* Medical green */
--color-warning: #f59e0b    /* Attention yellow */
--color-critical: #ef4444   /* Critical red */

👥 Team

🎓 EPITA Students - Google Healthcare Hackathon Team

Name	LinkedIn
Armand Blin
Arthur Courselle
Aurélien Daudin
Khaled Mili
Lucas Duport
Samy Yacef

🔮 Future Enhancements

• 🧠 Enhanced AI: Integration with more advanced MedGemma models
• 🔒 Security: HIPAA compliance and data encryption
• 📊 Analytics: Advanced patient insights and reporting
• 🌍 Internationalization: Multi-language support
• 📱 Mobile App: Native mobile applications
• 🔌 API Integration: Real EHR system connections

📄 License

This project is part of the Google Healthcare Hackathon and is intended for educational and demonstration purposes.

---

🏆 Proudly built for the Google Healthcare Hackathon 2025

Revolutionizing healthcare through AI-powered pre-diagnosis and virtual consultations