git.lucas.co / cce-vault
notes vault library and CLI (Obsidian-compatible)
git clone https://git.lucas.co/cce-vault.git

CLAUDE.md (5.7K)

  1 # cce-vault
  2 
  3 The notes vault shared by cce apps: a folder of Markdown notes, canvases
  4 and attachments, kept **byte-compatible with Obsidian** so the real app can
  5 keep working on the same files. This crate is everything about that folder
  6 that is not UI. It is milestone 1 of the Obsidian-on-cce plan; the apps
  7 that sit on it (`cce-notes`, a vault mode in `cce-graph`, note cards on
  8 `cce-grid`, vault tasks in `cce-list`) come later.
  9 
 10 It has **no cce-ui dependency** on purpose. The `cce-vault` CLI, tests and
 11 any non-GUI tool use it without a Wayland stack. There is **no daemon**:
 12 each app embeds an `Index` and a `VaultWatcher` and applies the watcher's
 13 batches on its own event loop. The files are the source of truth.
 14 
 15 ## Layout
 16 
 17 | Module | What it owns |
 18 | --- | --- |
 19 | `parse` | One note → properties, links, tags, headings, block ids, tasks, all with byte spans |
 20 | `index` | Every file, link resolution, backlinks, unresolved links, tags, tasks; the optional parse cache |
 21 | `watch` | Recursive `notify` watcher, debounced (150 ms quiet, 1 s cap), hidden paths dropped |
 22 | `search` | Fuzzy names (quick switcher), full-text scan, unlinked mentions |
 23 | `write` | `atomic_write`, create/append, `set_task`, `rename` with link rewrite |
 24 | `canvas` | JSON Canvas as `serde_json::Value`, written byte-exact with Obsidian |
 25 | `daily` | Daily notes from `.obsidian/daily-notes.json`; moment.js formats; template variables |
 26 | `config` | The vault root: `--vault`, `$CCE_VAULT`, then `vault { path "…" }` in `~/.config/cce/config.kdl` |
 27 | `main.rs` | The `cce-vault` CLI (`cce-vault --help`) |
 28 
 29 ## Invariants — each was a design decision, keep them
 30 
 31 - **Obsidian syntax is scanned from raw bytes; pulldown-cmark only marks
 32   code.** pulldown decides what is code/math and finds headings and inline
 33   `[text](dest)` links. Wikilinks, embeds, `#tags`, `^block` ids,
 34   `%%comments%%` and task statuses are scanned by hand outside those
 35   ranges. That gives every link an exact `target_span`, which is what a
 36   rename rewrites. pulldown's own `ENABLE_WIKILINKS` stays **off**: it knows
 37   neither `![[embed]]` nor the `\|` escape inside tables.
 38 - **Every write re-reads and re-parses the file it edits.** The index may
 39   be a watcher batch behind (Obsidian or a sync client wrote a second ago),
 40   and a span from a stale parse cuts the wrong bytes. `set_task` and
 41   `rename` both do this. Do not "optimise" it into using the index's copy.
 42 - **Resolution follows Obsidian**, case-insensitive throughout: an exact
 43   vault path first (relative to the note first for a *markdown* link; a
 44   leading `/` is always the vault root), then by file name with the link's
 45   folder part as a path suffix, preferring the linking note's own folder,
 46   then the shortest path, then alphabetical. Aliases do **not** resolve
 47   links, as in Obsidian. `[[Beta]]` and `[[Beta.md]]` both mean `Beta.md`;
 48   every other file keeps its extension (`[[pic.png]]`, `[[Board.canvas]]`).
 49 - **Relink is whole-vault after any change batch.** Adding `Beta.md` must
 50   turn every unresolved `[[Beta]]` anywhere into a backlink, and it costs
 51   ~21 ms for 100,000 links. Don't make it incremental without a benchmark.
 52 - **Hidden means ignored**: any path component starting with `.`
 53   (`.obsidian`, `.trash`, `.git`, and this crate's own `.name.PID.cce-tmp`
 54   write temps). The walk, `Index::rel` and the watcher all apply it.
 55   `.obsidian/` is read (daily-note settings) and **never written**.
 56 - **Canvases are `Value`s with `preserve_order`, never typed structs**, and
 57   `canvas::to_string` reproduces Obsidian's layout (tab indent, one compact
 58   node per line, no trailing newline). A rename that touches a canvas must
 59   diff as the one field it changed. The ignored test
 60   `real_canvases_round_trip` checks real files:
 61   `CCE_CANVAS_DIR=<vault> cargo test -- --ignored`.
 62 - **Rename writes links in the shortest form that still reaches the file**,
 63   and in full when the link was written in full or the short name would be
 64   captured by another file. `.md` suffixes, subpaths, display text and
 65   embed `!` are kept; markdown links stay relative or rooted as they were,
 66   and stay angle-bracketed or %-encoded as they were. A note moved to
 67   another folder gets its own relative links re-rooted, and any short
 68   wikilink whose target would change under the same-folder rule is pinned
 69   to its old target (`pin_moved_links`).
 70 
 71 ## Performance (measured 2026-09-30)
 72 
 73 A generated vault of 5,000 notes (40 MB, 100,000 links, 7,500 tasks) on the
 74 20-core laptop, warm page cache, release build:
 75 
 76 | Step | Time |
 77 | --- | --- |
 78 | Parse (8 threads) | 36 ms |
 79 | Relink (8 threads) | 21 ms |
 80 | Load the JSON parse cache instead of parsing | 47 ms (18 MB file) |
 81 | A whole CLI call (`backlinks`, `find`, `rename --dry-run`) | ~90 ms |
 82 | `search`, `mentions` on top of the open | +15 ms, +35 ms |
 83 
 84 So the parse cache (`Index::open(root, true)`, CLI `--cache`) is **opt-in**.
 85 It can only pay off where reading the files is the slow part (a cold page
 86 cache at login, a network filesystem), and that has not been measured.
 87 Regenerate the test vault with `bench/gen-vault.py <dir>`, then time
 88 `cce-vault --vault <dir> stats` (`RUST_LOG=debug` prints the phase split).
 89 It lives in `bench/`, not `scripts/`: ccebuild installs every crate's
 90 `scripts/` into `~/.local/bin`.
 91 
 92 ## Build and test
 93 
 94 ```sh
 95 cargo test -p cce-vault                  # unit tests, incl. a real notify watcher
 96 cargo build --release -p cce-vault
 97 cce-vault --vault <dir> stats            # or set CCE_VAULT
 98 ```
 99 
100 Test writes against a **copy** of a vault (`cp -a`), never the live one:
101 the live vault syncs to other devices.
102 
103 This crate is a workspace member and must still build standalone (its own
104 `Cargo.lock` is committed). Install the CLI with
105 `ccebuild install --no-build cce-vault` after a release build.