Give Coding Agents Your Project Docs With node:sqlite and FTS5

September 17, 2026 • 6 min read

On a reinsurance platform I built at an agency this year, the documentation got big. Requirements, specs, schema notes, meeting notes and client emails. It went from 65 documents in May to 291.

Coding agents had two ways to deal with that. Read loads of files at the start of every session, or guess. Neither is good.

On 9 May I asked an agent whether SQLite FTS5 with BM25 ranking would cut down the research phase at the start of each session. Forty minutes later the first version of a docs search CLI was in the repo.

It’s just a way to get context into a coding agent session via a CLI. It became one of the most used tools on the project.

What it turned into

The CLI stayed with the agency I built it for, so I can’t share its code. What it did:

search ranked matches across every document.
context returned the matching section plus the sections around it. This was the one agents used most.
facts pulled out decisions, requirements, risks and assumptions so they could be queried on their own.
Source priority ranked specs above schema notes, and those above meeting notes. Documents marked as superseded dropped down.
It read PDF and Word documents as well as Markdown.

Step one of the project’s agent instructions was to use it. Across my sessions, agents called it 1,122 times in 327 sessions, and context accounted for 615 of those.

It had no dependencies beyond Node. node:sqlite has FTS5 built in, so the whole thing is a database file and some SQL.

A minimal version

This is a small version I wrote from scratch for this post. It indexes a folder of Markdown, one chunk per heading, and has search and context commands:

1import { readdirSync, readFileSync } from 'node:fs';
2import { join, relative } from 'node:path';
3import { DatabaseSync } from 'node:sqlite';
4
5const db = new DatabaseSync('docs.db');
6
7db.exec(`
8	CREATE VIRTUAL TABLE IF NOT EXISTS chunks USING fts5(
9		path, heading, body, seq UNINDEXED
10	);
11`);
12
13function markdown_files(dir: string): string[] {
14	return readdirSync(dir, { recursive: true, encoding: 'utf8' })
15		.filter((file) => file.endsWith('.md'))
16		.map((file) => join(dir, file));
17}
18
19// One chunk per heading, so results point at a section, not a whole file
20function chunk(markdown: string) {
21	const chunks: { heading: string; body: string }[] = [];
22	let heading = '';
23	let lines: string[] = [];
24	const content = markdown.replace(/^---\n[\s\S]*?\n---\n/, '');
25	for (const line of content.split('\n')) {
26		if (/^#{1,3} /.test(line)) {
27			if (lines.join('').trim())
28				chunks.push({ heading, body: lines.join('\n') });
29			heading = line.replace(/^#+ /, '');
30			lines = [];
31		} else {
32			lines.push(line);
33		}
34	}
35	if (lines.join('').trim())
36		chunks.push({ heading, body: lines.join('\n') });
37	return chunks;
38}
39
40function index(dir: string) {
41	db.exec('DELETE FROM chunks');
42	const insert = db.prepare(
43		'INSERT INTO chunks (path, heading, body, seq) VALUES (?, ?, ?, ?)',
44	);
45	let count = 0;
46	for (const file of markdown_files(dir)) {
47		const path = relative(dir, file);
48		chunk(readFileSync(file, 'utf8')).forEach((c, seq) => {
49			insert.run(path, c.heading, c.body, seq);
50			count++;
51		});
52	}
53	console.log(`Indexed ${count} chunks`);
54}
55
56// bm25 weights: a match in the path or heading counts for more than the body
57function search(query: string, limit = 5) {
58	return db
59		.prepare(
60			`SELECT path, heading, seq,
61				snippet(chunks, 2, '[', ']', '…', 12) AS snippet
62			FROM chunks WHERE chunks MATCH ?
63			ORDER BY bm25(chunks, 3.0, 2.0, 1.0) LIMIT ?`,
64		)
65		.all(query, limit);
66}
67
68// The matched section plus the ones either side of it
69function context(query: string) {
70	const [top] = search(query, 1) as { path: string; seq: number }[];
71	if (!top) return 'No match';
72	return db
73		.prepare(
74			`SELECT heading, body FROM chunks
75			WHERE path = ? AND seq BETWEEN ? AND ? ORDER BY seq`,
76		)
77		.all(top.path, top.seq - 1, top.seq + 1)
78		.map((c) => `## ${c.heading}\n${c.body}`)
79		.join('\n');
80}
81
82const [command, arg] = process.argv.slice(2);
83if (command === 'index') index(arg);
84else if (command === 'search')
85	for (const r of search(arg))
86		console.log(
87			`${r.path} › ${r.heading}\n  ${String(r.snippet).replace(/\s+/g, ' ')}`,
88		);
89else if (command === 'context') console.log(context(arg));
90else
91	console.log(
92		'Usage: docs-search index <dir> | search <query> | context <query>',
93	);

Node 24 runs TypeScript directly, so there’s no build step. I pointed it at the posts on this blog:

1node docs-search.ts index ./posts
2# Indexed 2647 chunks
3
4node docs-search.ts search "hook instructions"

Results will come back something like this:

1how-to-make-claude-code-follow-hook-instructions.md › Hooks
2  …Execute [hook] [instructions] FIRST — before any reasoning, tool calls, or response text…
3how-to-make-claude-code-follow-hook-instructions.md › Receipts
4  …a on user submit prompt [hook] fired?" - **March 19** — "there's a…
5how-to-make-claude-code-follow-hook-instructions.md › I'm not the only one
6  …anthropics/claude-code/issues/18660) — "[Instructions] are read but not followed" - [#27032…

That’s about 260 posts indexed in under half a second, including Node starting up, and a search takes around 50ms. Every result comes back from the right post.

The bits that matter

Chunking by heading. An agent wants the section that answers its question, not a whole document. Splitting on headings means search returns something small enough to read, and the heading tells the agent what it’s looking at.

Weighted ranking. bm25(chunks, 3.0, 2.0, 1.0) gives the path three times the weight of the body, and the heading twice. A match in a file called authentication.md is a much stronger signal than the same word in passing somewhere else. You can see it in the output above: every result is from the post whose file name matches.

Context around the match. context returns the matched section with its neighbours. Docs tend to explain something across a couple of sections, and this is why it became the most used command on the real project.

Telling the agent to use it

A CLI only helps if the agent reaches for it. On the project it was the first step in the agent instructions, something like:

1Before starting work, search the project docs:
2
3- `node docs-search.ts context "<topic>"` for the relevant section
4- `node docs-search.ts search "<terms>"` to find related documents
5
6Cite the file and section you relied on.

I didn’t need an MCP server for any of this, which is why I’ve mostly stopped building them. Models are good with a CLI, and they can run --help when they’re unsure.

Where it went next

Working on that CLI changed how I thought about the docs folder. On 31 May I described it as going from a dumping ground to memory and context infrastructure.

That’s where wiki0 came from. It’s my open-source take on the same idea: Markdown as the source of truth, with a SQLite index that can be rebuilt any time for search, backlinks and facts. It’s still early.

If I were adding to the minimal version above, I’d start with the two things that made the biggest difference on the project: source priority, so a spec outranks a meeting note, and retrying with OR when a strict search finds too little.

There's a reactions leaderboard you can check out too.

Useful? Share it on Bluesky.

✨ View the stats for this post ✨

Sign up for the newsletter

Want to keep up to date with what I'm working on?

Join other developers and sign up for the newsletter.

I care about the protection of your data. Read the Privacy Policy for more info.