AI Note Taker: Automatic Notes from Meetings and Lectures
The hard part of meeting notes is retrieval months later: you captured points but not decisions, and the recording is lost. A local stack keeps audio on your machine. Four stages: record, transcribe, structure, index.
📜 Table of Contents
AI Note Taker: Automatic Notes from Meetings and Lectures, Locally
The hard part of meeting notes is retrieval months later. A local stack keeps audio on your machine; setup costs an hour or two.
Four stages
Record system audio → VAD trim (Silero) → transcribe (faster-whisper) → optional speaker separation (pyannote) → structure with a local model (Ollama) → index the archive.
Record the system output
For online meetings you must capture what the speakers send, not your microphone. On Windows use WASAPI loopback via soundcard; on macOS install BlackHole and a multi-output device; on Linux use a PulseAudio monitor source. For in-person sessions a phone or lavalier at 16kHz mono is enough.
Transcribe with faster-whisper
model = WhisperModel("large-v3", device="cuda", compute_type="float16")
segments, info = model.transcribe("meeting.m4a", language="zh", vad_filter=True,
beam_size=5, initial_prompt="Mandarin meeting about product, budget, schedule.")
for s in segments:
print(f"[{s.start:.1f}] {s.text}")
| Parameter | Effect | Advice |
|---|---|---|
| model | accuracy vs speed | large-v3 for Chinese; small for live |
| compute_type | numeric precision | float16 on GPU, int8 on CPU |
| vad_filter | skip silence | always on for long meetings |
| initial_prompt | domain vocabulary | put company, product and people names in, it fixes homophone errors |
| beam_size | search width | 5 is the balance |
The initial_prompt is the most overlooked setting and the most effective. One wrong proper noun poisons the structured output.
Structure with a hard template
Ask for fixed JSON fields and forbid inference. Require a timestamp on every decision and action so a human can re-listen and verify.
r = ollama.chat(model="qwen3:14b", format="json",
messages=[{"role": "user", "content": PROMPT + transcript}])
Archive and search
One Markdown file per meeting, named by date and topic, with frontmatter for attendees, project and tags. ripgrep is enough to start; move to Khoj or meilisearch when the archive grows. Searching "who said the budget was short" then lands on a file and a timestamp.
The fields that matter
Summary, decisions, actions with owner and due date, open questions, risks. An action without an owner and a date is not an action.
Privacy
Everything stays local unless you sync. Tell attendees you are recording; some jurisdictions require everyone's consent, and company policy may restrict recording on work devices.
FAQ
Chinese accuracy? large-v3 clearly beats small and medium; heavy accents degrade it, so seed domain terms via initial_prompt.
Speaker separation? pyannote.audio works but needs a HuggingFace token and model terms. Overlapping speech still fails; proofread important meetings.
Live captions? Yes, in 30-second chunks with a few seconds of latency. Use small live and re-run large-v3 afterwards for the archive.
Does the model invent action items? Sometimes. Forbid inference, require timestamps, and spot-check.
Note: model names and parameters change; see faster-whisper and Ollama docs.
Related
Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.