The author built a local transcription tool and fine-tuned Llama 3.2 3B to convert audio notes into structured JSON with strong accuracy, outperforming larger models in targeted evaluations.
Llama 3.2 3B fine-tuned with LoRA on RTX 4090/2070 GPUs, inference via llama.cpp (GGUF Q4_K_M) running in LM Studio. Training used Unsloth with synthetic + gold data pipeline.
2025-09-02
- website
Highlight 1
Delivers highly structured JSON outputs, making transcripts easier to integrate into productivity workflows.
Highlight 2
Local-first design ensures user privacy and independence from external APIs or cloud services.
Highlight 3
Fine-tuning significantly improves completeness and factual accuracy compared to baseline and competing models.
Improvement 1
The current solution requires technical setup (GPU, LM Studio, GGUF), which may limit accessibility for non-technical users.
Improvement 2
The user experience is primarily developer-oriented; a friendlier UI or packaged app could broaden adoption.
Improvement 3
Limited evaluation scope (100 samples) could be expanded for more robust performance validation across diverse inputs.
Product Functionality
Provide a pre-built installer or desktop app to simplify setup for non-technical users and possibly extend support to CPU-only environments.
UI & UX
Introduce a clean, user-friendly interface with drag-and-drop audio uploads and direct JSON export to reduce reliance on command-line use.
SEO or Marketing
Enhance discoverability with better documentation, case studies, and tutorials on practical use cases (e.g., journaling, meeting notes, task management).
MultiLanguage Support
Expand transcription and JSON structuring to handle multiple languages beyond English, enabling broader adoption internationally.
- 1
What does this tool do?
It transcribes audio notes locally and processes them into structured JSON with key details like title, tags, entities, dates, and actions.
- 2
Do I need internet access or external APIs to use it?
No, the tool runs fully locally using Whisper/Parakeet for transcription and a fine-tuned Llama model for structuring.
- 3
What hardware is required?
It runs best on GPUs such as RTX 4090 or 2070 Super, though performance varies depending on hardware capabilities.