Echoverse

Every sound has a universe hiding in it.

Echoverse records ambient sounds and generates fictional universes where those sounds are native. A coffee shop's clinking cups become crystalline chime-beetles in an underground bazaar. Traffic hum becomes the breath of colossal stone guardians.

Built entirely on Google AI — Gemini for imagination, Imagen 4 for visuals, Veo 3.1 for video.

How It Works

1. Record — capture 5-30 seconds of ambient sound
2. Analyze — Gemini 2.5 Flash identifies individual sound sources, spatial qualities, and atmosphere
3. Imagine — 3 parallel universe concepts are generated and auto-evaluated; the most creative wins
4. Generate — Imagen 4 renders the fictional world; Gemini verifies sound-visual alignment and refines if needed
5. Reveal — side-by-side: your reality vs. the Echoverse

Optional: generate an 8-second video with Veo 3.1, composited with your original audio.

Architecture

Browser (mic) → FastAPI → Gemini 2.5 Flash (analysis + world-building)
                        → Imagen 4 (reality + fantasy images)
                        → Gemini verification loop (score < 7 → regenerate)
                        → Veo 3.1 (optional video)
                        → ffmpeg (stitch + composite)

Key techniques:

• 3 world branches generated in parallel via ThreadPoolExecutor, auto-evaluated by Gemini
• Reality image + world exploration run concurrently
• Cross-modal verification: generated image fed back to Gemini to confirm sound-visual alignment
• Retry with exponential backoff on 429 rate limits

Project Structure

server.py              — FastAPI server, background generation pipeline
engine/
  pipeline.py          — Core AI pipeline (analyze, imagine, generate, verify)
  prompts.py           — Prompt templates for Gemini
  storage.py           — Dual backend: Firestore (cloud) / SQLite (local)
  gcs.py               — GCS file upload helper
static/
  index.html           — Single-page app (7 screens)
  app.js               — Frontend logic, Google SSO, polling
  style.css            — Dark theme UI
  favicon.svg          — App icon
deploy/
  setup-gcp.sh         — Enable APIs, create bucket/Firestore, grant IAM roles
  setup-sso.sh         — Instructions for creating OAuth client ID
  deploy.sh            — Deploy to Cloud Run
  map-domain.sh        — Map custom domain
Dockerfile             — Container definition

Local Development

Prerequisites: Python 3.13+, ffmpeg, a GCP project with Vertex AI enabled

# 1. Clone and install
pip install -r requirements.txt

# 2. Set up environment
cp .env.example .env
# Edit .env — set GOOGLE_CLOUD_PROJECT (for Vertex AI / Imagen 4)
# Or set GOOGLE_API_KEY for free-tier AI Studio (lower quality images)

# 3. Run
uvicorn server:app --host 0.0.0.0 --port 8080

# 4. Open http://localhost:8080

Auth is skipped when GOOGLE_CLIENT_ID is not set, so local dev works without SSO.

Cloud Deployment (GCP Cloud Run)

# 1. Set your project
export GOOGLE_CLOUD_PROJECT=your-project-id

# 2. Create GCP resources (bucket, Firestore, IAM)
bash deploy/setup-gcp.sh

# 3. Set up Google SSO (creates OAuth client ID)
bash deploy/setup-sso.sh

# 4. Deploy
export GOOGLE_CLIENT_ID=xxx.apps.googleusercontent.com
export ALLOWED_EMAILS=user1@gmail.com,user2@gmail.com  # empty = allow all
bash deploy/deploy.sh

# 5. (Optional) Map custom domain
bash deploy/map-domain.sh
# Then create CNAME: your-domain → ghs.googlehosted.com

Environment Variables

Variable	Required	Description
GOOGLE_CLOUD_PROJECT	Yes	GCP project ID (Vertex AI, Firestore, GCS)
GOOGLE_CLOUD_LOCATION	No	Region (default: us-central1)
GOOGLE_API_KEY	Alt	AI Studio API key (alternative to Vertex AI)
GCS_BUCKET	Cloud	GCS bucket for generated files
USE_FIRESTORE	Cloud	Set to 1 to use Firestore instead of SQLite
GOOGLE_CLIENT_ID	Cloud	OAuth client ID for Google SSO
ALLOWED_EMAILS	No	Comma-separated email allowlist (empty = allow all)

Google AI Models Used

Model	Purpose
Gemini 2.5 Flash	Audio analysis, world imagination, branch evaluation, image verification
Imagen 4	Reality and fantasy image generation (Vertex AI)
Veo 3.1	8-second video generation (optional)
gemini-2.0-flash-exp	Fallback image generation (AI Studio free tier)

Viewing Logs (Cloud Run)

# Stream live logs
gcloud run services logs tail echoverse --project=$GOOGLE_CLOUD_PROJECT --region=us-central1

# Recent logs
gcloud run services logs read echoverse --project=$GOOGLE_CLOUD_PROJECT --region=us-central1 --limit=100

# Or use Cloud Console:
# https://console.cloud.google.com/run/detail/us-central1/echoverse/logs