AI & Agents · Intermediate
Ask-a-Podcast RAG Search
Transcribe a podcast back-catalog, chunk and embed it, then answer questions with cited timestamps that deep-link into episodes.
You build a retrieval-augmented generation (RAG) pipeline that turns a podcast back-catalog into a searchable knowledge base. Audio episodes are transcribed with Whisper, split into overlapping chunks tied to their source timestamps, embedded into a vector store, and served through a FastAPI backend that retrieves relevant passages and feeds them to an LLM to answer natural-language questions. The frontend displays answers alongside cited timestamps that link directly to the moment in the episode, making the knowledge inside hundreds of hours of audio genuinely discoverable.
What you build
- Batch-transcribes MP3/MP4 episode files using faster-whisper, storing word-level timestamps alongside the text
- Chunks transcripts into overlapping segments and embeds them with a sentence-transformer model
- Stores and queries embeddings via pgvector (PostgreSQL) or Qdrant for fast semantic search
- FastAPI endpoint accepts a natural-language question, retrieves the top-K chunks, and streams an LLM-generated answer
- Every answer includes source citations showing episode title, chunk text, and a deep-link timestamp URL
- Next.js frontend provides a search bar, streaming answer display, and clickable episode cards
- Re-index endpoint lets you add new episodes without rebuilding the full corpus
What it teaches
- How to use Whisper and faster-whisper to transcribe audio with word-level timestamps
- RAG pipeline design: chunking strategy, embedding models, and vector store upsert patterns
- Semantic search with pgvector or Qdrant including index tuning and top-K retrieval
- Prompt construction for retrieval-augmented generation with cited sources
- Streaming LLM responses through FastAPI to a Next.js frontend
- Metadata filtering and provenance tracking so every generated claim links back to its source
How it works
- 1
Ingest
- MP3/MP4 files
- RSS feed
↓ transcribe
- 2
Transcribe
- faster-whisper
- word timestamps
↓ chunk
- 3
Chunk & Embed
- 300-token windows
- sentence-transformers
↓ upsert
- 4
Vector Store
- pgvector / Qdrant
- episode metadata
↓ retrieve
- 5
RAG Endpoint
- FastAPI
- LLM + context
↓ stream
- 6
UI
- Next.js
- timestamp deep-links
Sign in to open the build guide
Free account. Get the step-by-step build and every resource link.
Take it further
- Add speaker diarization using pyannote-audio so citations show who said what, not just when it was said.
- Build a scheduled ingestion cron job that polls an RSS feed, downloads new episodes automatically, transcribes and indexes them, then sends a Slack or email digest of newly searchable content.
- Implement hybrid search combining BM25 keyword matching with vector similarity (reciprocal rank fusion) to improve recall on proper nouns like guest names and product names.
