notes vault library and CLI (Obsidian-compatible)
git clone https://git.lucas.co/cce-vault.git
CLAUDE.md (5.7K)
1 # cce-vault
2
3 The notes vault shared by cce apps: a folder of Markdown notes, canvases
4 and attachments, kept **byte-compatible with Obsidian** so the real app can
5 keep working on the same files. This crate is everything about that folder
6 that is not UI. It is milestone 1 of the Obsidian-on-cce plan; the apps
7 that sit on it (`cce-notes`, a vault mode in `cce-graph`, note cards on
8 `cce-grid`, vault tasks in `cce-list`) come later.
9
10 It has **no cce-ui dependency** on purpose. The `cce-vault` CLI, tests and
11 any non-GUI tool use it without a Wayland stack. There is **no daemon**:
12 each app embeds an `Index` and a `VaultWatcher` and applies the watcher's
13 batches on its own event loop. The files are the source of truth.
14
15 ## Layout
16
17 | Module | What it owns |
18 | --- | --- |
19 | `parse` | One note → properties, links, tags, headings, block ids, tasks, all with byte spans |
20 | `index` | Every file, link resolution, backlinks, unresolved links, tags, tasks; the optional parse cache |
21 | `watch` | Recursive `notify` watcher, debounced (150 ms quiet, 1 s cap), hidden paths dropped |
22 | `search` | Fuzzy names (quick switcher), full-text scan, unlinked mentions |
23 | `write` | `atomic_write`, create/append, `set_task`, `rename` with link rewrite |
24 | `canvas` | JSON Canvas as `serde_json::Value`, written byte-exact with Obsidian |
25 | `daily` | Daily notes from `.obsidian/daily-notes.json`; moment.js formats; template variables |
26 | `config` | The vault root: `--vault`, `$CCE_VAULT`, then `vault { path "…" }` in `~/.config/cce/config.kdl` |
27 | `main.rs` | The `cce-vault` CLI (`cce-vault --help`) |
28
29 ## Invariants — each was a design decision, keep them
30
31 - **Obsidian syntax is scanned from raw bytes; pulldown-cmark only marks
32 code.** pulldown decides what is code/math and finds headings and inline
33 `[text](dest)` links. Wikilinks, embeds, `#tags`, `^block` ids,
34 `%%comments%%` and task statuses are scanned by hand outside those
35 ranges. That gives every link an exact `target_span`, which is what a
36 rename rewrites. pulldown's own `ENABLE_WIKILINKS` stays **off**: it knows
37 neither `![[embed]]` nor the `\|` escape inside tables.
38 - **Every write re-reads and re-parses the file it edits.** The index may
39 be a watcher batch behind (Obsidian or a sync client wrote a second ago),
40 and a span from a stale parse cuts the wrong bytes. `set_task` and
41 `rename` both do this. Do not "optimise" it into using the index's copy.
42 - **Resolution follows Obsidian**, case-insensitive throughout: an exact
43 vault path first (relative to the note first for a *markdown* link; a
44 leading `/` is always the vault root), then by file name with the link's
45 folder part as a path suffix, preferring the linking note's own folder,
46 then the shortest path, then alphabetical. Aliases do **not** resolve
47 links, as in Obsidian. `[[Beta]]` and `[[Beta.md]]` both mean `Beta.md`;
48 every other file keeps its extension (`[[pic.png]]`, `[[Board.canvas]]`).
49 - **Relink is whole-vault after any change batch.** Adding `Beta.md` must
50 turn every unresolved `[[Beta]]` anywhere into a backlink, and it costs
51 ~21 ms for 100,000 links. Don't make it incremental without a benchmark.
52 - **Hidden means ignored**: any path component starting with `.`
53 (`.obsidian`, `.trash`, `.git`, and this crate's own `.name.PID.cce-tmp`
54 write temps). The walk, `Index::rel` and the watcher all apply it.
55 `.obsidian/` is read (daily-note settings) and **never written**.
56 - **Canvases are `Value`s with `preserve_order`, never typed structs**, and
57 `canvas::to_string` reproduces Obsidian's layout (tab indent, one compact
58 node per line, no trailing newline). A rename that touches a canvas must
59 diff as the one field it changed. The ignored test
60 `real_canvases_round_trip` checks real files:
61 `CCE_CANVAS_DIR=<vault> cargo test -- --ignored`.
62 - **Rename writes links in the shortest form that still reaches the file**,
63 and in full when the link was written in full or the short name would be
64 captured by another file. `.md` suffixes, subpaths, display text and
65 embed `!` are kept; markdown links stay relative or rooted as they were,
66 and stay angle-bracketed or %-encoded as they were. A note moved to
67 another folder gets its own relative links re-rooted, and any short
68 wikilink whose target would change under the same-folder rule is pinned
69 to its old target (`pin_moved_links`).
70
71 ## Performance (measured 2026-09-30)
72
73 A generated vault of 5,000 notes (40 MB, 100,000 links, 7,500 tasks) on the
74 20-core laptop, warm page cache, release build:
75
76 | Step | Time |
77 | --- | --- |
78 | Parse (8 threads) | 36 ms |
79 | Relink (8 threads) | 21 ms |
80 | Load the JSON parse cache instead of parsing | 47 ms (18 MB file) |
81 | A whole CLI call (`backlinks`, `find`, `rename --dry-run`) | ~90 ms |
82 | `search`, `mentions` on top of the open | +15 ms, +35 ms |
83
84 So the parse cache (`Index::open(root, true)`, CLI `--cache`) is **opt-in**.
85 It can only pay off where reading the files is the slow part (a cold page
86 cache at login, a network filesystem), and that has not been measured.
87 Regenerate the test vault with `bench/gen-vault.py <dir>`, then time
88 `cce-vault --vault <dir> stats` (`RUST_LOG=debug` prints the phase split).
89 It lives in `bench/`, not `scripts/`: ccebuild installs every crate's
90 `scripts/` into `~/.local/bin`.
91
92 ## Build and test
93
94 ```sh
95 cargo test -p cce-vault # unit tests, incl. a real notify watcher
96 cargo build --release -p cce-vault
97 cce-vault --vault <dir> stats # or set CCE_VAULT
98 ```
99
100 Test writes against a **copy** of a vault (`cp -a`), never the live one:
101 the live vault syncs to other devices.
102
103 This crate is a workspace member and must still build standalone (its own
104 `Cargo.lock` is committed). Install the CLI with
105 `ccebuild install --no-build cce-vault` after a release build.