MCP Local RAG

🍴 Forked from shinpr/mcp-local-rag — original work by Shinsuke Kagawa

Local code intelligence engine for AI coding assistants. AST-level semantic chunking + keyword boost for pinpointing functions, classes, and APIs — fully private, zero setup.

📖 中文文档

---

Table of Contents

1. Features
2. Quick Start
   • 2.1 Configure Your AI Coding Tool
   • 2.2 CLI Quick Start
   • 2.3 First-Time Project Indexing
3. Core Concepts
   • 3.1 Dual-Strategy Chunking
   • 3.2 Hybrid Search
   • 3.3 Security Boundary
4. MCP Tool Reference
   • 4.1 Ingest Tools
   • 4.2 Search Tools
   • 4.3 Management Tools
   • 4.4 Code Intelligence
   • 4.5 System Tools
5. CLI
   • 5.1 Basic Commands
   • 5.2 CLI Configuration
6. Network & Models
   • 6.1 Mirror Auto-Detection
   • 6.2 Model Selection
   • 6.3 File Watching
7. Search Tuning
8. Performance Tuning
   • 8.1 Quantization Precision
   • 8.2 Execution Device
   • 8.3 Minimum Chunk Length
   • 8.4 Recommended Configurations
9. Configuration Reference
10. Troubleshooting
11. Development

---

1. Features

• Smart dual-strategy chunking — AST-level code chunking via tree-sitter (splits at function/class/method boundaries, injects scope chain + imports). Semantic chunking for documents (splits by meaning, not character count).
• Semantic search + keyword boost — Vector search first, then keyword matching boosts exact terms. useEffect, error codes, class names rank higher — not just semantically guessed.
• 15 MCP tools — Ingest, search, manage, code intelligence, and system ops in one server.
• AST code intelligence — find_definition and find_references for IDE-level code navigation, powered by tree-sitter metadata captured at ingest time.
• Three-tier mirror auto-fallback — huggingface.co → hf-mirror.com → modelscope.cn, zero config for users in mainland China.
• Runs entirely locally — No API keys, no cloud, no data leaving your machine. Works offline after the first model download.
• Zero-friction setup — One npx command. No Docker, Python, or servers to manage.

---

2. Quick Start

Set BASE_DIR to the folder you want to search (BASE_DIRS for multiple roots — see Configuration).

2.1 Configure Your AI Coding Tool

Cursor — ~/.cursor/mcp.json:

{
  "mcpServers": {
    "local-rag": {
      "command": "npx",
      "args": ["-y", "@damoqiongqiu/mcp-local-rag"],
      "env": { "BASE_DIR": "/path/to/your/project" }
    }
  }
}

Claude Code:

claude mcp add local-rag --scope user --env BASE_DIR=/path/to/your/project -- npx -y @damoqiongqiu/mcp-local-rag

Codex — ~/.codex/config.toml:

[mcp_servers.local-rag]
command = "npx"
args = ["-y", "@damoqiongqiu/mcp-local-rag"]

[mcp_servers.local-rag.env]
BASE_DIR = "/path/to/your/project"

WorkBuddy — Settings → Custom Connectors → Add:

{
  "mcpServers": {
    "local-rag": {
      "command": "npx",
      "args": ["-y", "@damoqiongqiu/mcp-local-rag"],
      "env": { "BASE_DIR": "/path/to/your/project" }
    }
  }
}

⚠️ WorkBuddy: you MUST click "Trust" in the Custom Connectors list after adding, otherwise the server is silently blocked.

2.2 CLI Quick Start

No MCP needed — run directly from the terminal:

npx @damoqiongqiu/mcp-local-rag ingest ./src/
npx @damoqiongqiu/mcp-local-rag query "auth middleware"
npx @damoqiongqiu/mcp-local-rag status

That's it. No Docker, Python, or server setup.

2.3 First-Time Project Indexing

You: "Index the src directory of this project"
Assistant: Successfully ingested 156 files (2,847 chunks created)

You: "Where's the middleware that handles API rate limiting?"
Assistant: src/middleware/rateLimiter.ts — useRateLimiter(), lines 42–89

You: "How is the database connection pool configured?"
Assistant: src/config/database.ts — createPool() default max: 20, idle: 5

---

3. Core Concepts

3.1 Dual-Strategy Chunking

Chunking strategy is chosen per file type:

• Code files (50+ languages) — CodeChunker parses source via tree-sitter AST, splits at structural boundaries (functions, classes, methods). Each chunk's contextualizedText includes its scope chain and import context for precise semantic search.
• Documents (PDF/DOCX/TXT/MD/HTML) — SemanticChunker splits into sentences, groups by embedding similarity to find natural topic boundaries. Markdown code blocks remain intact — never split mid-block.

3.2 Hybrid Search

Search = semantic similarity + keyword boost (RAG_HYBRID_WEIGHT, default 0.6):

1. Query vectorization → semantic search finds most relevant chunks
2. Quality filters apply (distance threshold, grouping)
3. Keyword matching boosts exact-term rankings

Exact identifiers like useEffect are never buried by semantic approximations.

3.3 Security Boundary

Only files under BASE_DIR / BASE_DIRS are accessible for ingest, list, delete, or read-neighbor operations. Symlinks resolved outside roots are rejected. Sibling-prefix paths (e.g., /foo/barista when root is /foo/bar) are also blocked — prevents path traversal attacks.

---

4. MCP Tool Reference

15 tools organized into 5 categories.

4.1 Ingest Tools

#	Tool	Purpose	Example
1	ingest_file	Single file (PDF/DOCX/TXT/MD/code)	"Ingest ./docs/api-spec.pdf"
2	ingest_data	In-memory text/HTML	"Fetch this page and ingest the HTML"
3	ingest_directory	Bulk directory ingest	"Ingest everything under ./src"

ingest_file supports 50+ code languages. PDFs support an optional visual mode — a local VLM generates captions for figure pages, making visual content searchable. Two profiles available:

Profile	Model	Cache	Suited for
fast (default)	SmolVLM-256M	~250 MB	Light visual indexing
quality	Qwen2.5-VL-3B-ONNX	~2.9 GB	Figures with in-image text

# CLI
npx @damoqiongqiu/mcp-local-rag ingest ./spec.pdf --visual --visual-quality quality
# MCP
"Ingest ./spec.pdf with visual: true, visualQuality: 'quality'"

ingest_data runs Readability → Markdown → index. Perfect for web content fetched by your AI assistant. Re-ingesting replaces old versions automatically.

ingest_directory scans recursively, respects .gitignore, shows real-time progress via MCP notifications.

4.2 Search Tools

#	Tool	Purpose	Key Parameters
4	query_documents	Hybrid search (semantic + keyword)	query, limit, scope, highlightContext, fromTimestamp
5	read_chunk_neighbors	Expand context around results	filePath, chunkIndex, before, after

query_documents — scope accepts a single path prefix or list, restricting results to that subtree. highlightContext returns snippets around matched terms. fromTimestamp / untilTimestamp enable time-range filtering.

read_chunk_neighbors — defaults to 2 chunks before and after (like grep -C 2), max 50 each. Response includes the target chunk marked isTarget: true.

4.3 Management Tools

#	Tool	Purpose
6	list_files	List files with ingestion status (ingested: true/false)
7	delete_file	Delete by file path or source URL
8	status	Index stats: docs, chunks, memory, search mode

list_files supports scope filtering with the same prefix-match semantics as search. In large directories, scope accelerates the scan by skipping out-of-scope subtrees.

4.4 Code Intelligence

#	Tool	Purpose	Input
9	find_definition	Locate symbol definition (file, line range, scope)	Exact symbol name
10	find_references	Find all references (import + text mention)	Symbol name

Both tools depend on AST metadata (imports, entities, scope chains) extracted by tree-sitter at ingest time. Only works for code files ingested with CodeChunker — files ingested before v0.18.7 lack this metadata and require reindex_all to rebuild.

find_references uses a two-phase strategy: (1) exact match in codeMeta.imports → (2) FTS full-text search for the symbol name. Results are deduplicated by (filePath, chunkIndex), with import references listed first.

4.5 System Tools

#	Tool	Purpose
11	config	Runtime hot read/write config — no restart needed
12	dedup_check	SHA256 + Jaccard similarity to detect duplicate files
13	export_index	Export entire index as JSON (backup or migration)
14	reindex_all	Full re-chunk + re-embed (after model change)
15	reindex_stale	Re-ingest only files modified on disk (incremental sync)

config hot-swaps hybridWeight, modelName, cacheDir, baseDir/baseDirs, etc. Switching models auto-disposes the old Embedder and initializes the new one — note: changing models alters the embedding space and requires reindex_all.

dedup_check is especially useful in monorepos — spot ↔ futures mirror code is typically flagged with similarity 1.0.

---

5. CLI

5.1 Basic Commands

# Ingest
npx @damoqiongqiu/mcp-local-rag ingest ./src/

# Search (with scope)
npx @damoqiongqiu/mcp-local-rag query "auth middleware"
npx @damoqiongqiu/mcp-local-rag query "auth" --scope /docs/api

# Context expansion
npx @damoqiongqiu/mcp-local-rag read-neighbors --file-path /abs/path.md --chunk-index 5

# Management
npx @damoqiongqiu/mcp-local-rag list --scope /docs/api
npx @damoqiongqiu/mcp-local-rag status
npx @damoqiongqiu/mcp-local-rag delete ./docs/old.pdf
npx @damoqiongqiu/mcp-local-rag delete --source "https://..."

query, read-neighbors, list, status, delete emit JSON to stdout (pipe to jq). ingest emits progress to stderr.

Global options (--db-path, --cache-dir, --model-name) go before the subcommand:

npx @damoqiongqiu/mcp-local-rag --help

⚠️ The CLI does NOT read your MCP client config (mcp.json, etc.). Configure via flags or environment variables.

5.2 CLI Configuration

Flags — global options before, subcommand options after:

npx @damoqiongqiu/mcp-local-rag --db-path ./my-db query "auth" --base-dir ./docs

--base-dir is repeatable on ingest and list:

npx @damoqiongqiu/mcp-local-rag ingest --base-dir ./docs --base-dir ./specs ./docs/readme.md

Environment variables:

export DB_PATH=./my-db
export BASE_DIR=./docs
npx @damoqiongqiu/mcp-local-rag query "auth"

For multiple roots, use BASE_DIRS (JSON array):

export BASE_DIRS='["/Users/me/work","/Users/me/specs"]'

Precedence: CLI flags > environment variables > defaults.

---

6. Network & Models

6.1 Mirror Auto-Detection

huggingface.co is inaccessible from mainland China. Built-in three-tier mirror chain with automatic fallback:

huggingface.co → hf-mirror.com → modelscope.cn

At startup, each mirror is HEAD-probed (3s timeout). The first reachable mirror with a complete API is selected:

• With proxy (HTTPS_PROXY) → direct to huggingface.co
• No proxy → auto-switch to hf-mirror.com
• hf-mirror API unavailable → fallback to modelscope.cn

No manual HF_ENDPOINT required. For manual control:

Env Var	Effect
HF_AUTO_MIRROR=false	Disable auto-detection, use huggingface.co only
HF_ENDPOINT=<url>	Force a specific mirror, skip auto-detection

v0.18.5+ uses setGlobalDispatcher(ProxyAgent) — all Node.js 22 network requests go through the proxy.

6.2 Model Selection

6 embedding models with alias resolution via model-registry:

Model	Alias	Size	Dims
Xenova/all-MiniLM-L6-v2 (default)	mini	~90 MB	384
Xenova/all-MiniLM-L12-v2	—	~120 MB	384
Xenova/bge-small-en-v1.5	bge-small	~130 MB	384
Xenova/all-mpnet-base-v2	mpnet	~420 MB	768
Xenova/bge-base-en-v1.5	—	~420 MB	768
Xenova/multi-qa-mpnet-base-dot-v1	multi-qa	~420 MB	768

Guidance: code repos → default model + high keyword boost; multilingual → consider embeddinggemma-300m; scientific papers → consider allenai-specter.

RAG_DTYPE controls ONNX precision (fp32 / fp16 / q8). Default fp32; use q8 when memory-constrained. ⚠️ Changing models or dtype requires deleting DB_PATH and re-indexing.

6.3 File Watching

Set RAG_WATCH=true — the server starts recursive fs.watch on baseDirs (500ms debounce):

• File creation/modification → auto ingest_file
• File deletion → auto delete_file

Ideal for actively changing projects.

---

7. Search Tuning

Variable	Default	Description
RAG_HYBRID_WEIGHT	0.6	Keyword boost: 0 = semantic only, 1 = keyword only
RAG_GROUPING	unset	similar = top group only, related = top 2 groups
RAG_MAX_DISTANCE	unset	Filter low-relevance results (e.g., 0.5)
RAG_MAX_FILES	unset	Limit results to top N files

Code-focused tuning (recommended default):

{ "RAG_HYBRID_WEIGHT": "0.7", "RAG_GROUPING": "similar" }

Document-focused tuning:

{ "RAG_HYBRID_WEIGHT": "0.4", "RAG_GROUPING": "related" }

Keyword boost is applied after semantic filtering — improves precision without introducing noise.

---

8. Performance Tuning

Beyond search accuracy, inference performance is also configurable. All optimizations are environment variables — no code changes required.

8.1 Quantization Precision (RAG_DTYPE)

Controls ONNX model inference precision. For all-MiniLM-L6-v2, three levels are available:

Value	Model Size	Speed	Memory	Precision Loss	Best For
fp32 (default)	~90 MB	baseline	~80 MB	none	First use, maximum accuracy
fp16	~45 MB	20-30% faster	~45 MB	negligible	Recommended for daily use
q8	~45 MB	30-50% faster	~45 MB	minor	Low memory, large projects

"env": { "RAG_DTYPE": "fp16", "BASE_DIR": "..." }

⚠️ Changing dtype requires index rebuild — embedding spaces are incompatible.

Verify it works: After restart, call status via MCP and check the dtype field. Should match your setting (e.g., "fp16").

If it fails: Startup throws EmbeddingError with a list of supported dtypes. Common cause: the model doesn't provide the q8 variant — switch to fp16.

8.2 Execution Device (RAG_DEVICE)

Controls which ONNX Runtime backend to use:

Value	Backend	Notes
cpu (default)	CPU	Most stable, no extra dependencies
webgpu	GPU (WebGPU)	⚠️ Experimental: M1/M2 Mac uses Metal, NVIDIA uses Vulkan

"env": { "RAG_DEVICE": "webgpu", "RAG_DTYPE": "fp16", "BASE_DIR": "..." }

⚠️ Changing device changes the embedding space — requires index rebuild. Stacks with RAG_DTYPE — fp16 + webgpu gives both model-size reduction and GPU speedup.

Verify it works: MCP startup log should show Loading model on device "webgpu". status should show device: "webgpu".

If it fails:

• Unsupported device at startup → WebGPU unavailable in your environment, revert to "cpu"
• Starts successfully but inference crashes → likely an ONNX WebGPU backend bug, revert to "cpu"
• Just delete the RAG_DEVICE line to fall back — other config is untouched

8.3 Minimum Chunk Length (CHUNK_MIN_LENGTH)

Filters out chunks shorter than this value during ingest. Default 50 keeps nearly everything; 200 drops 30-40% of noise fragments.

"env": { "CHUNK_MIN_LENGTH": "200", "BASE_DIR": "..." }

⚠️ Blunt instrument — short but important code (e.g., config constants) may also be discarded. Requires index rebuild. Sweet spot: 100-200.

8.4 Recommended Configurations

Scenario	Config
Daily development	RAG_DTYPE=fp16
Large project + M1/M2 Mac	RAG_DTYPE=fp16, RAG_DEVICE=webgpu
Memory-constrained	RAG_DTYPE=q8

All changes require reindex_all (MCP) or re-running ingest (CLI). If something breaks, delete the failing env line to revert to defaults.

---

9. Configuration Reference

MCP server: environment variables only (via your MCP client's env block). CLI: environment variables + equivalent flags (flags take precedence).

Env Var	CLI Flag	Default	Description
BASE_DIR	--base-dir (repeatable)	cwd	Document root (security boundary)
BASE_DIRS	—	unset	JSON array of roots, overrides BASE_DIR
DB_PATH	--db-path	./lancedb/	Vector database path
CACHE_DIR	--cache-dir	./models/	Model cache — recommend absolute path
MODEL_NAME	--model-name	all-MiniLM-L6-v2	HuggingFace model ID
MAX_FILE_SIZE	--max-file-size	100 MB	Max file size in bytes
CHUNK_MIN_LENGTH	--chunk-min-length	50	Min chunk length (1–10000 chars)
RAG_DEVICE	—	cpu	ONNX execution device
RAG_DTYPE	—	fp32	Quantization (fp32/fp16/q8)
HTTPS_PROXY	—	unset	Model download proxy. v0.18.5+ globally effective
HF_ENDPOINT	—	huggingface.co	Manual mirror override
HF_AUTO_MIRROR	—	true	Auto-detection toggle
RAG_WATCH	—	unset	File watching (true/1)

Root resolution order: CLI --base-dir > BASE_DIRS > BASE_DIR > cwd. BASE_DIRS and BASE_DIR are never merged. Only JSON array syntax supported for BASE_DIRS — delimiter syntax is intentionally rejected.

---

10. Troubleshooting

Model download failed

Symptoms: fetch failed, status shows searchMode: fts instead of hybrid.

Solutions:

1. Network restriction (mainland China, etc.) — use proxy:

   "env": { "HTTPS_PROXY": "http://127.0.0.1:7890" }
   

   Set in your MCP client config, not the terminal. v0.18.5+ globally effective via setGlobalDispatcher.

2. Auto-mirror fallback (v0.18.2+, default) — three-tier probe. Usually works without any config.

3. Manual override — HF_ENDPOINT=https://modelscope.cn or download models manually into CACHE_DIR.

4. npx cached old version — clear and restart:

   rm -rf ~/.npm/_npx/
   

MCP client doesn't see tools

1. Verify config file syntax
2. WorkBuddy users: confirm "Trust" button clicked
3. Restart client completely (Cmd+Q on macOS)
4. Test directly: npx @damoqiongqiu/mcp-local-rag should run without errors

Rebuilding the index

After switching models or when the database is corrupted:

1. Stop the MCP service
2. Delete DB_PATH directory (default ./lancedb/) — safe, doesn't affect source files
3. Restart MCP → fresh database auto-created
4. Bulk re-ingest:

   npx @damoqiongqiu/mcp-local-rag ingest ./src/
   

FAQ

• Private? Yes. After model download, nothing leaves your machine.
• Offline? Yes, once models are cached.
• Supported formats? 50+ code languages + PDF/DOCX/TXT/MD/HTML. No Excel, PPT, or images.
• GPU acceleration? Opt-in via RAG_DEVICE. Support depends on your system, Node.js version, and the ONNX backend.
• Backup? Copy the DB_PATH directory.

---

11. Development

git clone https://github.com/damoqiongqiu/mcp-local-rag.git
cd mcp-local-rag
pnpm install

pnpm test              # All tests
pnpm run type-check    # TypeScript check
pnpm run check:fix     # Lint + format
pnpm run check:all     # Full CI pipeline

src/
  index.ts      # Entry point
  server/       # MCP tool handlers
  cli/          # CLI subcommands
  parser/       # PDF/DOCX/TXT/MD/code parsing
  chunker/      # SemanticChunker + CodeChunker
  embedder/     # Transformers.js embeddings
  vectordb/     # LanceDB operations
  utils/        # Shared utilities (security, scan, scope)
  __tests__/    # Test suites

---

License

MIT License. Free for personal and commercial use.

Acknowledgments

Built with Model Context Protocol (Anthropic), LanceDB, and Transformers.js.