notes editor over the vault (Obsidian-compatible)
git clone https://git.lucas.co/cce-notes.git
feat: show embedded images (![[pic.png]]) in reading and live preview
images.rs loads them for both views: a lookup by link text that the
cache cannot answer is recorded, resolved after the frame through the
vault's link rules, decoded on a thread (png/jpeg/gif/webp/bmp via
image, svg via resvg at its own size; big rasters scaled to 2048 px) and
uploaded when it returns, then the layouts are redone so the link becomes
the picture. A vault change re-resolves links and redecodes a changed
image; renderer_init forgets ids after a reconnect (checked at scale 2
with CCE_UI_FAULT_RECONNECT against a control without it).
Needs cce-ui 9d5568e and cce-vault 97626a1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CLAUDE.md | 35 +++++++-
Cargo.toml | 4 +
src/images.rs | 257 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
src/main.rs | 64 ++++++++++++++-
4 files changed, 355 insertions(+), 5 deletions(-)
diff --git a/CLAUDE.md b/CLAUDE.md
index 2c70e4a..5d99aa6 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -25,6 +25,7 @@ proposal".
| `side.rs` | What the panes list: backlinks + unlinked mentions, outline, text and tag search |
| `complete.rs` | `[[` completion: the open link at the caret, the shortest link text, the splice |
| `mcp.rs` | MCP tools (`search`, `find_notes`, `read_note`, `backlinks`, `open_note`, `append_daily`, `current_note`) |
+| `images.rs` | Embedded images: link text → vault path → decode thread → upload; the lookup the reading view and the editor share |
| `instance.rs` | Single instance on `/tmp/cce-notes-<WAYLAND_DISPLAY>.sock`; the CLI's commands |
## Behaviour worth knowing before changing it
@@ -74,6 +75,34 @@ proposal".
Wayland modifiers event) — a key event's own `ctrl` flag is stale for the
Ctrl press itself.
+## Embedded images (`images.rs`)
+
+A paragraph that is one `![[pic.png]]` / `` draws as the
+picture in reading and in live preview (cce-ui's `markdown::layout_with`
+and `DocEditor::set_images`; Obsidian's `|300` / `|300x200` sizes
+honoured, too-wide images scaled to the column). Inline embeds inside a
+sentence and note embeds (`![[Note]]`) still show as links.
+
+- **Asked for while painting, loaded after.** A lookup the cache cannot
+ answer is only recorded; `Images::pump`, at the end of `display_list`,
+ resolves it with `Index::resolve_text` from the open note and decodes on
+ a thread. The decode returns as `Message::ImageDecoded` (which wakes an
+ idle loop) and is uploaded there; then the reading layout and the
+ editor's line layouts are dropped so the link becomes the picture.
+- **A link that resolves to an image already decoded still needs a
+ relayout** — it drew as a link this frame. `pump` says so and the app
+ sends itself `Message::ImagesReady`. Without it, every vault change (all
+ links resolve afresh) turned loaded images back into links.
+- **Ids die with the renderer.** `renderer_init` forgets them all on the
+ second and later renderer (`seen_renderer`). Verified with
+ `CCE_UI_FAULT_RECONNECT` at scale 2 against a control built without it:
+ the control's images went blank, these stayed.
+- Rasters over 2048 px are scaled down on decode (reported at their own
+ size, so sizing is unchanged); SVGs show at their intrinsic size,
+ rasterised at twice it. Formats: png, jpeg, gif (first frame), webp,
+ bmp, svg — not avif. The 32 most recently drawn stay decoded across
+ notes.
+
## The reading view (`reading.rs`)
cce-ui's text prim draws one run in one style, so a paragraph is laid out
@@ -148,9 +177,9 @@ scale-1 shadow (1280×720) and pointer/caret maths at scale 2.
In the editor: property values are edited as raw YAML (the table flips
to raw when the caret enters; Obsidian edits in place), tables and
-callouts show raw, embeds (`![[…]]`) do not render,
-and a fenced block has no language label or copy button. Also heading
+callouts show raw, note embeds (`![[Note]]`) and embeds inside a sentence
+show as links (standalone image embeds render), and a fenced block has no language label or copy button. Also heading
completion (`[[Note#`), rendered snippets in the panes (they show
raw lines), search debounce for large vaults (it scans every note per
-keystroke), embedded images, the icon (`Icon=cce-notes` has no SVG in
+keystroke), the icon (`Icon=cce-notes` has no SVG in
cce-icons yet), and per-note scroll in the history.
diff --git a/Cargo.toml b/Cargo.toml
index f236a1a..c03dd5b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -13,6 +13,10 @@ chrono = "0.4"
log = "0.4"
serde_json = "1"
env_logger = "0.11"
+# Embedded images: rasters decoded off the UI thread; SVG through the
+# same resvg cce-ui draws icons with (its version, so one copy builds).
+image = { version = "0.25", default-features = false, features = ["png", "jpeg", "gif", "webp", "bmp"] }
+resvg = "0.41.0"
[dev-dependencies]
tempfile = "3"
diff --git a/src/images.rs b/src/images.rs
new file mode 100644
index 0000000..cd33f22
--- /dev/null
+++ b/src/images.rs
@@ -0,0 +1,257 @@
+//! Embedded images (`![[pic.png]]`, ``) for the reading view
+//! and the live-preview editor.
+//!
+//! Both ask by an embed's **link text** while they lay out and paint
+//! ([`Images::lookup`], shared with the editor through an `Rc`). A link not
+//! seen yet is only recorded; after the frame, [`Images::pump`] resolves it
+//! the way every vault link resolves (`Index::resolve_text`, relative to the
+//! open note), and an image file is decoded on a thread. The decode comes
+//! back as [`crate::Message::ImageDecoded`] — which also wakes an idle loop —
+//! and [`Images::decoded`] uploads it; the host then lays out again so the
+//! link becomes the picture. A decode that fails leaves the link showing.
+//!
+//! Ids belong to one renderer: [`Images::renderer_reset`] forgets every one
+//! after a reconnect and the next frame asks again. Decoded images are kept
+//! for the last [`KEEP`] used, so walking back and forth between notes does
+//! not decode the same picture twice.
+
+use std::cell::RefCell;
+use std::collections::{HashMap, HashSet};
+use std::path::{Path, PathBuf};
+use std::rc::Rc;
+
+use cce_ui::widget::EmbedImage;
+use cce_vault::Index;
+
+/// Decoded images kept across notes.
+const KEEP: usize = 32;
+/// The largest texture side: a column is under 800 logical px, 1600 at
+/// output scale 2, so more is memory nobody sees.
+const MAX_TEX: u32 = 2048;
+
+/// A decoded image, straight RGBA8, and the size it shows at before any
+/// fitting — its own pixels for a raster (even when the texture was made
+/// smaller), its intrinsic size for an SVG.
+#[derive(Clone)]
+pub struct Decoded {
+ pub pixels: Vec<u8>,
+ pub tex: (u32, u32),
+ pub natural: (u32, u32),
+}
+
+impl std::fmt::Debug for Decoded {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ write!(f, "Decoded {{ tex: {:?}, natural: {:?} }}", self.tex, self.natural)
+ }
+}
+
+enum Entry {
+ Loading,
+ Failed,
+ Ready { image: EmbedImage, used: u64 },
+}
+
+#[derive(Default)]
+struct Inner {
+ /// Link text → vault path, for the open note (`None`: not an image, or
+ /// it resolves nowhere). Cleared when the note or the vault changes.
+ links: HashMap<String, Option<String>>,
+ /// Link texts asked for that `links` does not know yet.
+ asked: HashSet<String>,
+ entries: HashMap<String, Entry>,
+ clock: u64,
+}
+
+#[derive(Clone, Default)]
+pub struct Images(Rc<RefCell<Inner>>);
+
+impl Images {
+ /// The uploaded image an embed's link text shows, if it is ready.
+ pub fn lookup(&self, link: &str) -> Option<EmbedImage> {
+ let mut inner = self.0.borrow_mut();
+ let path = match inner.links.get(link) {
+ Some(Some(p)) => p.clone(),
+ Some(None) => return None,
+ None => {
+ inner.asked.insert(link.to_string());
+ return None;
+ }
+ };
+ inner.clock += 1;
+ let now = inner.clock;
+ match inner.entries.get_mut(&path) {
+ Some(Entry::Ready { image, used }) => {
+ *used = now;
+ Some(*image)
+ }
+ _ => None,
+ }
+ }
+
+ /// Resolve what was asked for since the last pump and start decoding
+ /// the images among it. `done` carries a decode back to the app. True
+ /// when a link resolved to an image already decoded: it showed as a
+ /// link this frame, and the host must lay out again to show it.
+ pub fn pump(&self, index: &Index, current: Option<&str>, done: impl Fn(String, Option<Decoded>) + Clone + Send + 'static) -> bool {
+ let mut inner = self.0.borrow_mut();
+ let asked: Vec<String> = inner.asked.drain().collect();
+ let mut ready = false;
+ for link in asked {
+ let path = index.resolve_text(current, &link).filter(|p| cce_vault::markdown::is_image(p));
+ inner.links.insert(link, path.clone());
+ let Some(path) = path else { continue };
+ if let Some(e) = inner.entries.get(&path) {
+ ready |= matches!(e, Entry::Ready { .. });
+ continue;
+ }
+ inner.entries.insert(path.clone(), Entry::Loading);
+ let abs = index.abs(&path);
+ let done = done.clone();
+ std::thread::spawn(move || {
+ let decoded = decode(&abs);
+ if decoded.is_none() {
+ log::warn!("could not decode {}", abs.display());
+ }
+ done(path, decoded);
+ });
+ }
+ ready
+ }
+
+ /// A decode finished: upload it. True when the image is new to the
+ /// screen (the host lays out again).
+ pub fn decoded(&self, path: String, decoded: Option<Decoded>) -> bool {
+ let mut inner = self.0.borrow_mut();
+ // Dropped (vault change, renderer reset) while it decoded.
+ if !matches!(inner.entries.get(&path), Some(Entry::Loading)) {
+ return false;
+ }
+ let Some(d) = decoded else {
+ inner.entries.insert(path, Entry::Failed);
+ return false;
+ };
+ let id = cce_ui::vk::upload_rgba_mipmapped(d.pixels, d.tex.0, d.tex.1);
+ inner.clock += 1;
+ let used = inner.clock;
+ let image = EmbedImage { id, width: d.natural.0, height: d.natural.1 };
+ inner.entries.insert(path, Entry::Ready { image, used });
+ evict(&mut inner);
+ true
+ }
+
+ /// The open note changed: its links resolve afresh.
+ pub fn note_changed(&self) {
+ self.0.borrow_mut().links.clear();
+ }
+
+ /// Files changed in the vault: links resolve afresh (an image may have
+ /// appeared), and a changed image decodes again. True when an image
+ /// on screen went away.
+ pub fn vault_changed(&self, index: &Index, paths: &[PathBuf]) -> bool {
+ let mut inner = self.0.borrow_mut();
+ inner.links.clear();
+ let mut dropped = false;
+ for p in paths {
+ let Some(rel) = index.rel(p) else { continue };
+ if let Some(Entry::Ready { image, .. }) = inner.entries.remove(&rel) {
+ cce_ui::vk::free_image(image.id);
+ dropped = true;
+ }
+ }
+ dropped
+ }
+
+ /// A new renderer: every id is dead (nothing to free). Forget them all.
+ pub fn renderer_reset(&self) {
+ let mut inner = self.0.borrow_mut();
+ inner.entries.clear();
+ inner.links.clear();
+ }
+}
+
+/// Free the least recently drawn images past [`KEEP`].
+fn evict(inner: &mut Inner) {
+ let mut ready: Vec<(u64, String)> = inner
+ .entries
+ .iter()
+ .filter_map(|(p, e)| match e {
+ Entry::Ready { used, .. } => Some((*used, p.clone())),
+ _ => None,
+ })
+ .collect();
+ if ready.len() <= KEEP {
+ return;
+ }
+ ready.sort();
+ for (_, path) in &ready[..ready.len() - KEEP] {
+ if let Some(Entry::Ready { image, .. }) = inner.entries.remove(path) {
+ cce_ui::vk::free_image(image.id);
+ }
+ }
+}
+
+/// Read and decode an image file (off the UI thread).
+pub fn decode(path: &Path) -> Option<Decoded> {
+ let bytes = std::fs::read(path).ok()?;
+ let is_svg = path.extension().and_then(|e| e.to_str()).is_some_and(|e| e.eq_ignore_ascii_case("svg"));
+ if is_svg {
+ return decode_svg(&bytes);
+ }
+ let img = image::load_from_memory(&bytes).ok()?;
+ let natural = (img.width(), img.height());
+ let img = if natural.0.max(natural.1) > MAX_TEX {
+ img.resize(MAX_TEX, MAX_TEX, image::imageops::FilterType::Triangle)
+ } else {
+ img
+ };
+ let rgba = img.into_rgba8();
+ let tex = rgba.dimensions();
+ Some(Decoded { pixels: rgba.into_raw(), tex, natural })
+}
+
+/// An SVG shows at its own size; it is rasterised at twice that (sharp at
+/// output scale 2), within [`MAX_TEX`].
+fn decode_svg(bytes: &[u8]) -> Option<Decoded> {
+ let tree = resvg::usvg::Tree::from_data(bytes, &resvg::usvg::Options::default(), cce_ui::widget::get_font_db()).ok()?;
+ let size = tree.size();
+ let natural = (size.width().round().max(1.0) as u32, size.height().round().max(1.0) as u32);
+ let px = (natural.0.max(natural.1) * 2).min(MAX_TEX);
+ let (pixels, w, h) = cce_ui::rasterize_svg(bytes, px)?;
+ Some(Decoded { pixels, tex: (w, h), natural })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn decodes_and_shrinks_big_rasters() {
+ let dir = tempfile::tempdir().unwrap();
+ let small = dir.path().join("s.png");
+ image::RgbaImage::new(30, 20).save(&small).unwrap();
+ let d = decode(&small).unwrap();
+ assert_eq!((d.tex, d.natural), ((30, 20), (30, 20)));
+ assert_eq!(d.pixels.len(), 30 * 20 * 4);
+ let big = dir.path().join("b.png");
+ image::RgbaImage::new(4096, 1024).save(&big).unwrap();
+ let d = decode(&big).unwrap();
+ assert_eq!((d.tex, d.natural), ((2048, 512), (4096, 1024)));
+ assert!(decode(&dir.path().join("missing.png")).is_none());
+ }
+
+ #[test]
+ fn decodes_svg_at_its_own_size() {
+ let dir = tempfile::tempdir().unwrap();
+ let p = dir.path().join("a.svg");
+ std::fs::write(&p, r#"<svg xmlns="http://www.w3.org/2000/svg" width="40" height="20"><rect width="40" height="20" fill="red"/></svg>"#).unwrap();
+ let d = decode(&p).unwrap();
+ assert_eq!((d.natural, d.tex), ((40, 20), (80, 40)));
+ }
+
+ #[test]
+ fn lookups_are_recorded_until_pumped() {
+ let images = Images::default();
+ assert_eq!(images.lookup("a.png"), None);
+ assert!(images.0.borrow().asked.contains("a.png"));
+ }
+}
diff --git a/src/main.rs b/src/main.rs
index 8def49d..0a4568c 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -19,6 +19,7 @@
//! `vault { path "…" }` in `~/.config/cce/config.kdl`. The app runs single
//! instance (see [`instance`]); `cce-notes <note>` opens a note in it.
+mod images;
mod instance;
mod reading;
mod tree;
@@ -79,6 +80,10 @@ pub enum Message {
Command(Command),
VaultChanged(Vec<PathBuf>),
Mcp(cce_ui::mcp::McpToolCall),
+ /// An embedded image finished decoding (vault path, pixels or failure).
+ ImageDecoded(String, Option<images::Decoded>),
+ /// Links resolved to images already decoded: lay out again.
+ ImagesReady,
Exit,
}
@@ -212,6 +217,12 @@ struct NotesApp {
hover_hit: bool,
editor: DocEditor,
+ /// Embedded images, shared with the editor (see `images.rs`).
+ images: images::Images,
+ sender: calloop::channel::Sender<Message>,
+ /// A renderer has been made before: the next one is a reconnect, and
+ /// every image id died with the old one.
+ seen_renderer: bool,
/// The editor's revision when its text last matched the disk.
saved_rev: u64,
/// The revision as last seen by the autosave check, and when it changed.
@@ -348,6 +359,7 @@ impl NotesApp {
/// Show `text` as the current note's content, as on disk.
fn load_text(&mut self, text: String) {
+ self.images.note_changed();
self.blocks = cce_vault::markdown::blocks(&text);
// Clears the undo history too: undo must not step back into another
// note's (or the pre-reload) text.
@@ -595,6 +607,13 @@ impl NotesApp {
fn vault_changed(&mut self, paths: Vec<PathBuf>) {
let Some(ix) = self.index.as_mut() else { return };
ix.apply_changes(&paths);
+ // An image added, changed or removed: links resolve again, and the
+ // editor asks again (it only asks while laying a line out).
+ let images_touched = paths.iter().any(|p| cce_vault::markdown::is_image(&p.to_string_lossy()));
+ self.images.vault_changed(ix, &paths);
+ if images_touched {
+ self.editor.invalidate();
+ }
self.rebuild_rows();
// Link colours, backlinks and search hits depend on every file.
self.layout = None;
@@ -1013,7 +1032,8 @@ impl NotesApp {
}
_ => true,
};
- let laid = reading::layout(&self.blocks, width, &self.theme, m, &resolved);
+ let images = &self.images;
+ let laid = reading::layout_with(&self.blocks, width, &self.theme, m, &resolved, &|t| images.lookup(t));
self.layout = Some((width, laid));
}
@@ -1277,7 +1297,8 @@ impl NotesApp {
}
let note = m.note;
if let Some((_, l)) = &self.layout {
- pc.clip(note, |pc| l.paint(pc, (ox, oy), self.read_scroll, note));
+ let images = &self.images;
+ pc.clip(note, |pc| l.paint_with(pc, (ox, oy), self.read_scroll, note, &|t| images.lookup(t)));
}
}
}
@@ -1456,7 +1477,10 @@ impl Application for NotesApp {
}
let _ = sender.send(Message::Command(startup.command.clone()));
+ let images = images::Images::default();
let mut editor = DocEditor::new("", EditorTheme::new(READ_SIZE), true);
+ let shared = images.clone();
+ editor.set_images(Box::new(move |t| shared.lookup(t)));
editor.max_width = READ_MAX_W;
editor.pad = READ_PAD;
let switcher_input = TextBox::new(String::new()).with_placeholder("Find or create a note…");
@@ -1487,6 +1511,9 @@ impl Application for NotesApp {
saved_rev: editor.buf.revision,
edit_seen: editor.buf.revision,
editor,
+ images,
+ sender: sender.clone(),
+ seen_renderer: false,
edit_changed_at: Instant::now(),
history: Vec::new(),
hist_pos: 0,
@@ -1533,6 +1560,16 @@ impl Application for NotesApp {
Message::Command(cmd) => self.run_command(cmd),
Message::VaultChanged(paths) => self.vault_changed(paths),
Message::Mcp(call) => self.mcp_call(call),
+ Message::ImageDecoded(path, decoded) => {
+ if self.images.decoded(path, decoded) {
+ self.layout = None;
+ self.editor.invalidate();
+ }
+ }
+ Message::ImagesReady => {
+ self.layout = None;
+ self.editor.invalidate();
+ }
Message::Exit => *_exit = true,
}
*needs_rebuild = true;
@@ -1631,6 +1668,17 @@ impl Application for NotesApp {
}
self.paint_status(&mut pc);
self.paint_switcher(&mut pc);
+ // What this frame asked for and nobody has: resolve and decode it.
+ if let Some(ix) = &self.index {
+ let tx = self.sender.clone();
+ let ready = self.images.pump(ix, self.current.as_deref(), move |path, d| {
+ let _ = tx.send(Message::ImageDecoded(path, d));
+ });
+ if ready {
+ // Through the loop, so a frame follows.
+ let _ = self.sender.send(Message::ImagesReady);
+ }
+ }
Some(pc.finish())
}
@@ -1638,6 +1686,18 @@ impl Application for NotesApp {
true
}
+ /// A reconnect builds a new renderer and every uploaded image id dies
+ /// with the old one: forget them, and the next frame uploads afresh.
+ /// The first renderer needs nothing (uploads queued before it drain
+ /// into it).
+ fn renderer_init(&mut self, _renderer: &mut cce_ui::vk::VkRenderer) {
+ if std::mem::replace(&mut self.seen_renderer, true) {
+ self.images.renderer_reset();
+ self.layout = None;
+ self.editor.invalidate();
+ }
+ }
+
/// Ctrl+Z reaches the editor through here: the runner routes the undo
/// chord to the focused widget first, and the editor is not one.
fn undo(&mut self, needs_rebuild: &mut bool) -> bool {