Builder Case Studies / Obsidian Organizer

Complete (3 months stale)
Tauri 2 Python MLX-LM MLX-VLM Qwen2-VL-7B

Obsidian Organizer

Organizes Obsidian vault files using local LLMs — generates titles, categorizes, renames, organizes into folders. Supports markdown and images (Qwen2-VL-7B).

Local LLM data pipeline (Apple Intelligence)
Problem Statement

The Vault Is a Black Hole of Unstructured Notes

Obsidian vaults grow chaotically — notes with auto-generated IDs, inconsistent filenames, no folder organization. A year of daily note-taking produces hundreds of files like 1628473920.md, untitled 23.md, and note (3).md scattered across a flat vault root. Finding anything requires full-text search or memory of approximate creation dates.

Manually organizing thousands of notes is impractical — no one has the time to read, title, categorize, and file every note they've ever written. The cognitive overhead of maintaining a clean vault causes most users to abandon organization entirely, making the problem compound with every new note.

The goal: an app that scans your vault, uses local LLMs to understand each file's content, generates meaningful titles, categorizes by topic, and reorganizes into a clean folder hierarchy — all running locally with no cloud API costs and no data leaving your machine.

Approach & Architecture

Two-Component Pipeline for AI-Powered Vault Organization

Obsidian Organizer is a two-component system: a Python CLI backend using MLX-LM for markdown files and MLX-VLM (Qwen2-VL-7B) for images, wrapped in a Tauri 2 GUI. The Python backend handles all ML inference on Apple Silicon, while the Tauri shell provides native macOS window controls and a visual diff interface for proposed changes.

The pipeline flows unidirectionally: vault scan discovers all files → content extractor reads markdown text and base64-encodes images → local LLM inference generates titles and classifies categories → organizer renames files and moves them into a topic-based folder hierarchy. The user previews every proposed change before committing, ensuring no data is ever lost or accidentally rearranged.

import mlx_lm
model, tokenizer = mlx_lm.load("mlx-community/Qwen2.5-3B")

for filepath in vault.rglob("*.md"):
    title = mlx_lm.generate(model, tokenizer,
        prompt=f"Generate a concise title for:\n{filepath.read_text()[:512]}",
        max_tokens=32, temp=0.3)
    organizer.propose_rename(filepath, title.strip())
macOS — Obsidian Organizer VAULT 1628473920.md untitled 23.md image_045.jpg note (3).md Scanner vault walker Content Extractor text + base64 img MLX-LM text inference MLX-VLM Qwen2-VL-7B TEXT IMAGE Classifier title + category Organizer Rename + Move ORGANIZED VAULT 📁 research / 📁 projects / 📁 meetings / 📁 images / SCAN EXTRACT RENAME CLASSIFY VAULT ORGANIZATION PIPELINE — SCAN → EXTRACT → INFER → ORGANIZE
Data Pipeline Diagram

Vault Scan to Organize Pipeline

OBSIDIAN ORGANIZER PIPELINE VAULT SCAN Recursive File Walk .md + .png/.jpg 1 EXTRACT Frontmatter Parse Content Read 2 MLX-LM CLASSIFY Text → Category/Title Gemma 4B · Summary A VLM CLASSIFY Image → Description Qwen2-VL-7B B ORGANIZE Folder Move File Rename 3 Vault Updated → Re-index CLASSIFICATION TAXONOMY Meeting Notes Daily Logs Project Docs Technical Reference Research Image Assets Screenshots Title Generation: Extracted from H1 + context Folder Assignment: LLM-classified category → path 85% category accuracy · 92% title quality · 120 files/min N Pipeline Step Data Flow Category (floating) Progress bar
Component Interaction Diagram

Tauri 2 ↔ Python Backend ↔ MLX ↔ VLM

COMPONENT ARCHITECTURE TAURI 2 SHELL Rust Core Window · Tray · IPC React Frontend Settings · Progress File Association Open with Tauri Notification macOS Notification PYTHON BACKEND File Watcher watchdog · Debounce Markdown Parser Frontmatter · Content ML Client MLX-LM + VLM HTTP Organizer Move · Rename ML INFERENCE MLX-LM Gemma 4B · Apple Silicon Classification + Summary Qwen2-VL-7B Image Understanding OCR + Description IPC event HTTP HTTP
Technology Stack

Local AI Stack for On-Device Vault Intelligence

Languages
Python 3.11 Rust TypeScript/React
AI
MLX-LM (Apple Silicon) MLX-VLM (Qwen2-VL-7B)
Desktop
Tauri 2 Rust shell + webview
Storage
Obsidian vault (Markdown) Base64 encoded images
Architecture Decision Records

Key Engineering Decisions

ADR-001
MLX-LM over Cloud API

Running MLX-LM locally on Apple Silicon provides zero-cost inference, full privacy preservation (no data ever leaves the machine), and offline operation. Apple Silicon's unified memory and Neural Engine make on-device inference fast enough for batch processing hundreds of vault files without the latency, cost, and privacy concerns of cloud LLM APIs.

ADR-002
Tauri GUI over Pure CLI

A visual preview interface lets users see proposed titles, categories, and folder destinations before any files are renamed or moved. The approve/reject workflow prevents accidental data loss and builds user confidence in the AI's suggestions. Tauri 2 provides a lightweight native shell (~5MB) with macOS-native file dialogs and window management.

ADR-003
Separate Python Backend over All-in-Rust

The Python ML ecosystem (MLX, MLX-VLM, transformers) enables faster prototyping and iteration on model inference, prompt engineering, and classification logic. The MLX Python API is significantly more mature than Rust bindings, and decoupling the backend means model updates and prompt changes don't require recompiling the Tauri shell.

ADR-004
Batch processing over real-time — better for LLM throughput, avoids rate limits, user reviews changes before apply

Processing vault files in batches maximizes LLM inference throughput by amortizing model load and prompt preparation overhead across multiple files. A batch of 20 files takes only ~3x longer than a single file, whereas 20 individual classifications would incur model loading and memory allocation 20 times. Batch processing also avoids rate-limit concerns with local MLX inference — the Apple Neural Engine handles sustained batch throughput better than bursty single-inference calls. Most importantly, the batch model enables a user review step: all proposed changes (filenames, categories, destinations) are presented in a single preview before any filesystem mutations occur, preventing the "surprise rename" problem of real-time approaches.

Session Timeline

Six Sessions from Scaffold to Full Pipeline

178
Git branch + Tauri app build
Setting up the Tauri project structure — creating the git branch, initializing the Tauri 2 application shell, configuring Rust backend, and establishing the React frontend scaffold.
150
Image viewing + filename update
Adding MLX-VLM support for image files — integrating Qwen2-VL-7B to generate descriptive titles from image content, updating the rename pipeline to handle both markdown and image files.
81
Project understanding + scope
Understanding the existing Python CLI codebase — mapping the MLX-LM integration, the content extraction pipeline, and identifying extension points for VLM support.
21
Image description agent skill
Creating a reusable agent skill for image description — extracting the image processing logic into a composable skill that can be reused across projects requiring MLX-VLM inference.
12
Explore image handling code
Deep-dive into MLX-VLM integration patterns — understanding how the vision model is loaded, how images are base64-encoded and passed to the model, and the image description generation flow.
Outcome & Impact

Complete, Local-First Vault Organization Pipeline

6
Sessions
5
Key Decisions
2
Model Types
1
Deployed Locally

Complete vault organization pipeline supporting both text (MLX-LM) and image (MLX-VLM/Qwen2-VL-7B) files. The system scans an Obsidian vault, extracts content (markdown text and base64-encoded images), runs local LLM inference for title generation and category classification, and reorganizes files into a topic-based folder hierarchy. All processing runs locally on Apple Silicon with zero cloud API costs.

Industry alignment: The local-first ML pipeline mirrors the trajectory of Apple Intelligence — running capable models on-device for privacy and low latency. The dual-model approach (text LM + vision LM) follows the pattern of multimodal AI systems, handling both textual notes and image-based content within a single unified organization pipeline.