crawl4ai for video & audio. One command turns any YouTube video, podcast episode, or local recording into clean, timestamped, LLM-ready markdown — or a TTS/STT training dataset of sliced audio clips paired with verbatim transcripts. Captions-first, runs locally, no plumbing.
One input — a link or a file — and two kinds of output. Read it (RAG, notes, agents) or train on it (text-to-speech, speech recognition):
📄 Clean markdown — for RAG, notes & agents
|
🎙️ TTS/STT dataset — for training
|
uv tool install hearsay
hearsay "https://youtu.be/VIDEO_ID" # → markdown
hearsay dataset "https://youtu.be/VIDEO_ID" --out ./data # → TTS/STT datasetCaptions when they exist (fast, no download); local Whisper or Apple-Silicon Parakeet transcription when they don't. Single videos, whole playlists, and podcast feeds. Nothing leaves your machine.
- How it works · Install
- 🎙️ Build TTS/STT datasets · 📄 Clean markdown
- Web UI · Transcription engines · MCP server
- How it compares · CLI reference · Requirements
One pipeline, two outputs. hearsay gets a word-timestamped transcript the cheapest way it can — existing captions if the source has them, otherwise local transcription — then either reflows it into readable markdown or slices the audio into training clips:
flowchart LR
S["YouTube · podcast feed<br/>playlist · local file"] --> T{"captions?"}
T -- yes --> C["fetch captions<br/>(no download)"]
T -- no --> W["transcribe locally<br/>Whisper / Parakeet"]
C --> X(["timestamped transcript<br/>+ word timings"])
W --> X
X --> M["📄 markdown<br/>paragraphs · timestamps<br/>chapters · JSON sidecar"]
X --> D["🎙️ TTS/STT dataset<br/>audio clips + transcripts<br/>LJSpeech · NeMo · HF"]
- Captions-first. Uses the source's captions when available — fast, no media download.
- Falls back to transcription automatically (CPU Whisper, or Parakeet on Apple Silicon).
- Local & private. Everything runs on your machine; hearsay hosts nothing and ships no data.
- Scales. One video, a whole YouTube playlist, or a podcast RSS feed — batched into one output.
uv tool install hearsay # recommended
# or
pipx install hearsayOptional extras:
uv tool install "hearsay[parakeet]" # fast Apple-Silicon transcription (macOS arm64)
uv tool install "hearsay[diarize]" # speaker diarization → single-voice TTS datasets
uv tool install "hearsay[mcp]" # MCP server, for AI agentsSystem requirement: ffmpeg on your PATH.
From source (for development)
git clone https://github.com/mudassar531/hearsay
cd hearsay
uv sync && uv run hearsay --help # or: uv tool install .hearsay dataset turns spoken media into a machine-learning training dataset:
the audio is sliced into short clips on word-level timestamps — never mid-word —
each paired with its exact, verbatim transcript and timing, in the standard layouts
training pipelines read directly.
# A short video → a dataset folder (LJSpeech metadata.csv + NeMo manifest.jsonl + wavs/)
hearsay dataset "https://youtu.be/VIDEO_ID" --out ./voice-data
# A whole playlist / channel / podcast feed → one merged dataset
hearsay dataset "https://example.com/feed.xml" --out ./speech-data
# 16 kHz mono for ASR, custom clip length, also emit a HuggingFace audiofolder index
hearsay dataset talk.mp3 --sample-rate 16000 --segment-min 2 --segment-max 12 --format hfYou get a portable folder, ready to point a trainer at:
voice-data/
wavs/VIDEO_ID_0001.wav … # mono 16-bit PCM, cut on sentence/pause boundaries, never mid-word
metadata.csv # LJSpeech: id|text|text (Coqui / Piper read this directly)
manifest.jsonl # NeMo / ESPnet: {"audio_filepath","duration","text","offset"}
metadata.jsonl # (with --format hf) HuggingFace audiofolder index
dataset_card.md # provenance, counts, language + a rights/consent note
dropped.jsonl # every filtered-out clip, with the reason
- Word-accurate, click-free cuts. Clips are sliced on word-level timestamps
(faster-whisper
word_timestamps, or Parakeet on Apple Silicon), padded a little on each edge (--pad) and given a short fade so boundaries never click or clip a phoneme. - Quality filtering (on by default) drops junk — too short/long, internal silence,
wrong-script or odd speaking-rate text, repetition, low ASR confidence — and logs
every drop with its reason to
dropped.jsonl.--no-filterkeeps everything. - Single-voice TTS from multi-speaker audio. Install
hearsay[diarize], accept the pyannote model conditions and setHF_TOKEN, then--dominant-speaker(keep the main voice) or--per-speaker(one index per speaker). Without it, datasets are mixed-speaker (fine for STT) — and the card says so. --normalizeloudness-normalizes each clip (two-pass EBU R128); builds over playlists/feeds are resumable.
Accuracy & rights. Word boundaries from Whisper/Parakeet are good but not phonetically exact — clips are padded and snapped to pauses, and you should spot-check. You are responsible for the rights to any media you process and for voice consent (cloning a real person's voice may require it); extracting audio from YouTube may breach its Terms. hearsay is local and ships no datasets. Informational, not legal advice — see each generated
dataset_card.md.
Want to see the output shape without running anything? There's a tiny committed
example under examples/dataset/.
Getting a transcript into a RAG pipeline usually means gluing together yt-dlp,
Whisper, and a pile of timestamp-wrangling scripts — and you still end up with one
line per caption fragment or an undifferentiated wall of text. hearsay does the whole
thing in one command, and the markdown is readable by a human and a model:
# YouTube → markdown via captions (fast — no download)
hearsay "https://www.youtube.com/watch?v=VIDEO_ID"
# Local audio/video → markdown (fast Parakeet on Apple Silicon, else CPU Whisper)
hearsay talk.mp3
# Force local transcription, pick an engine, also emit a JSON sidecar
hearsay "https://youtu.be/VIDEO_ID" --transcribe --model parakeet --json
# A podcast feed or YouTube playlist (list, or batch with --all)
hearsay "https://example.com/feed.xml" --all --limit 3 --output-dir ./outThe output: real paragraphs (not one line per caption), a [hh:mm:ss] timestamp on
each, and chapters as ## sections (or ~5-minute windows when there are none):
---
title: "You Would Be a Terrible Leader"
source: "https://www.youtube.com/watch?v=rStL7niR7gs"
channel: "CGP Grey"
duration: "00:18:13"
ingested: "2026-06-13T10:00:00Z"
method: "captions"
language: "en"
---
# You Would Be a Terrible Leader
## [00:00:00 – 00:05:21]
**[00:00:00]** Do you want to rule? Do you see the problems in your country and
know how to fix them? If only you had the power to do so. Well. You've come to
the right place. But, before we begin this lesson in political power, ask
yourself, why don't rulers see as clearly as you...The method field records exactly how the text was produced — captions,
captions-auto, whisper-small, parakeet-tdt-0.6b-v3 — so a consumer can tell a
human transcript from a machine one. Pass --json for a sidecar matching the
Transcript schema: metadata plus chunks[], each with
start_s, end_s, section, and text — ready to embed. Music or a song? Add
--no-vad so the vocals aren't filtered out as "non-speech."
Prefer a browser? hearsay web starts a tiny local web UI — paste a URL or drop in a
file, pick a model, and watch clean markdown render live with copy, download, and a
history. Tick Dataset to build a training dataset
instead and download it as a .zip. It's a single self-contained page on the Python
standard library — no extra dependencies — bound to 127.0.0.1, so nothing leaves
your machine.
hearsay web # → http://localhost:8756
hearsay web --port 9000 # custom port
hearsay web --host 0.0.0.0 # expose on your LAN (unauthenticated — careful)
Single video URLs and file uploads go through the UI; for playlists and podcast feeds, use the CLI (the UI shows a friendly hint).
When a source has no captions (or you pass --transcribe), hearsay transcribes
locally with the fastest engine your machine has. --model auto (the default) picks:
| Engine | When | Speed | Notes |
|---|---|---|---|
| Parakeet (NVIDIA Parakeet-TDT on Apple MLX) | Apple Silicon + parakeet extra |
~24× realtime (M1 Pro) | parakeet is multilingual (25 European languages); parakeet-en is English-only |
| Whisper (faster-whisper, CPU int8) | everywhere else, or an explicit size | ~7× realtime | sizes tiny…large-v3; large-v3 is the multilingual ceiling |
On Apple Silicon, Parakeet is about 3× faster than whisper-small at comparable
accuracy. If the parakeet extra isn't installed, auto falls back to whisper-small
automatically — so hearsay behaves the same everywhere, just faster on a Mac. Models
download once (Whisper: tens of MB to ~1.5 GB; Parakeet v3: ~2.5 GB) and cache for
offline use.
Speech vs. music: hearsay is tuned for spoken audio (podcasts, talks, interviews, meetings), where transcription is accurate. For music, pass
--no-vadso the vocals aren't discarded — but expect a rough lyric transcript, since these are speech models.
hearsay ships an MCP server so AI agents can ingest
media themselves. It exposes two tools — ingest_url(url, transcribe?, lang?) and
ingest_file(path) — that each return clean, timestamped markdown.
uv tool install "hearsay[mcp]"
hearsay mcp # stdio MCP server (Ctrl-C to stop)Claude Code:
claude mcp add hearsay -- hearsay mcpor add to .mcp.json (project) / ~/.claude.json (user):
{
"mcpServers": {
"hearsay": {
"type": "stdio",
"command": "hearsay",
"args": ["mcp"]
}
}
}Claude Desktop — add to claude_desktop_config.json (Settings → Developer → Edit
Config; macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\).
If hearsay isn't on the host's PATH, use the absolute path (which hearsay), or
"command": "python", "args": ["-m", "hearsay", "mcp"].
Server configuration (env vars, since MCP tool signatures are fixed):
| Variable | Default | Effect |
|---|---|---|
HEARSAY_MODEL |
auto |
auto, parakeet, parakeet-en, or a Whisper size (tiny…large-v3) |
HEARSAY_LANG |
(unset) | Default language: English captions, else transcription auto-detect |
HEARSAY_VAD |
1 |
Voice-activity filter (Whisper); set 0 for music/songs |
HEARSAY_PARAKEET_MODEL |
(unset) | Override the Parakeet MLX repo id (advanced) |
| hearsay | DIY yt-dlp + Whisper |
markitdown / docling | |
|---|---|---|---|
| Input | video & audio | video & audio (you wire it) | documents (pdf/docx/pptx) |
| One command | ✅ | ❌ multi-step plumbing | ✅ (for docs) |
| TTS/STT dataset export | ✅ LJSpeech + NeMo + HF, filtered, diarizable | ✗ DIY plumbing | ✗ |
| Captions-first (no download) | ✅ | ✗ usually re-transcribes | n/a |
| Timestamps + paragraph grouping | ✅ readable | ✗ raw segments | n/a |
| Chapters → sections | ✅ | ✗ manual | n/a |
| Podcasts · playlists · batch | ✅ | ✗ manual | ✗ |
| Fast Apple-Silicon engine | ✅ Parakeet (MLX) | ✗ DIY | n/a |
| JSON sidecar for RAG | ✅ stable schema | ✗ manual | varies |
| Browser UI + MCP server | ✅ | ✗ | varies |
hearsay does media; document tools like markitdown and docling do documents. Use both.
hearsay <SOURCE> [options] SOURCE = YouTube video/playlist URL, podcast RSS, or local file
-o, --output PATH Output file for a single source (default ./<id>.md)
--output-dir PATH Output directory for batch (playlist/feed) ingestion (default ./hearsay-out)
--lang CODE Language: captions default to English; transcription auto-detects
--transcribe Force local transcription even when captions exist
--model MODEL auto (default) | parakeet | parakeet-en | tiny | base | small | medium | large-v3
--no-vad Disable voice-activity filtering (Whisper; use for music/songs)
--json Also write a .json sidecar (Transcript schema)
--latest Batch: ingest only the most recent item
--episode N Batch: ingest only item N (1-indexed)
--all [--limit N] Batch: ingest all items (optionally capped)
hearsay dataset <SOURCE> [options] Build a TTS/STT training dataset
--out PATH Dataset output directory (default ./hearsay-dataset)
--format FMT ljspeech | jsonl | hf (repeatable; default ljspeech + jsonl)
--sample-rate HZ Output WAV rate (default 22050; 16000 for ASR)
--segment-min/max S Clip length bounds in seconds (default 1–15)
--pad S Edge padding added to each side of a clip (default 0.1)
--normalize EBU R128 loudness-normalize each clip
--no-filter Keep every clip (skip the quality filters)
--diarize Label speakers (needs hearsay[diarize] + HF_TOKEN)
--per-speaker Diarize and emit a per-speaker index
--dominant-speaker Diarize and keep only the most-spoken speaker
--model / --lang / --vad / --no-vad Transcription (as above)
--limit N Batch: cap items from a playlist/feed
hearsay web Run the local web UI (--host, --port)
hearsay mcp Run the MCP stdio server
hearsay --version Show the version
- Python 3.11+
- ffmpeg on your PATH. hearsay decodes most audio/video directly (faster-whisper bundles its own decoder), but ffmpeg is the safe baseline, slices dataset clips, and handles some yt-dlp format merges.
| OS | Install ffmpeg |
|---|---|
| macOS (Homebrew) | brew install ffmpeg |
| Debian / Ubuntu | sudo apt install ffmpeg |
| Fedora | sudo dnf install ffmpeg |
| Arch | sudo pacman -S ffmpeg |
| Windows (winget) | winget install Gyan.FFmpeg |
| Windows (Chocolatey) | choco install ffmpeg |
The first transcription downloads the chosen model once, then caches it for offline use.
See CONTRIBUTING.md and the good first issues. Changes are documented in CHANGELOG.md. hearsay does one thing well — turn media into clean markdown and training-ready datasets — and aims to keep doing exactly that.