web browser (Servo)
git clone https://git.lucas.co/cce-browser.git
feat(raindrop): bookmark sync, phase 3 — the browser syncs on its own
With browser.raindrop on, a worker thread (cce-raindrop) polls the
in-memory bookmarks every 2s, syncs a change once it holds still for a
poll, does a full pass every 10 minutes, and runs one on request from
cce://bookmarks. A pass's local half is one locked edit
(Bookmarks::edit_rows), skipped when the plan changes nothing here.
cce://bookmarks shows the last pass and "sync now"; a refused pass
shows why and "sync anyway", which carries a random code checked by the
handler, so no web page linking to cce:// can force a mass deletion.
The code holds while passes keep being refused — a fresh one each time
made the link on screen stale, found in the end-to-end run.
Also: Raindrop titles are flattened where they arrive (a tab or newline
would corrupt bookmarks.tsv and read as a rename every pass), and a
link edit skipped because the bookmark changed mid-pass leaves the base
rather than later trashing Raindrop's copy. CCE_RAINDROP_API points the
client at a stand-in for testing.
Run end to end in a shadow session against a stand-in Raindrop and a
throwaway keyring: import, local remove → trash, remote rename and add,
an emptied collection refused, sync anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CLAUDE.md | 16 ++-
RAINDROP-SYNC.md | 62 +++++++++--
src/main.rs | 16 +++
src/pages.rs | 144 +++++++++++++++++++++++-
src/raindrop/api.rs | 31 +++++-
src/raindrop/mod.rs | 54 +++++++--
src/raindrop/sync.rs | 310 +++++++++++++++++++++++++++++++++++++++++++++++++++
src/settings.rs | 6 +
8 files changed, 608 insertions(+), 31 deletions(-)
diff --git a/CLAUDE.md b/CLAUDE.md
index f1bd257..7669ad8 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -29,6 +29,7 @@ Sixteen files, ~8.8k lines. The ten that carry the design:
| `src/settings.rs` | the per-app KDL config |
| `src/accounts.rs` | accounts from cce-secrets: the Secret Service worker, which entries a host earns, saving a new login, and the never-save list |
| `src/wpe/formwatch.rs` | the page half of account autocomplete: the watcher every frame runs (fields, frame-offset relay, fill asks, sign-in capture) and the events it sends |
+| `src/raindrop/` | bookmark sync with Raindrop.io's Unsorted: the three-way merge (`mod.rs`), the REST client (`api.rs`), the worker and status line (`sync.rs`) — design in RAINDROP-SYNC.md |
## Build
@@ -534,6 +535,19 @@ run it `--foreground` in the background so one process owns the bus name.
`examples/wpe_autofill.rs` covers the engine side (frames, relay, fill, submit,
focus gating) with no keyring at all.
+## Raindrop bookmark sync
+
+With `browser.raindrop` on, bookmarks sync with Raindrop.io's **Unsorted**
+collection: a worker thread polls the in-memory bookmarks, runs a three-way
+merge against `raindrop-sync.tsv` (the pairs as of the last pass), and shows
+its status on `cce://bookmarks`. **Read `RAINDROP-SYNC.md` before touching
+`src/raindrop/`** — every rule in the merge (identity by Raindrop id, only link
+and title ever sent, the deletion guard, the base recording what *happened*)
+exists because the alternative silently deletes or duplicates bookmarks. The
+token is a keyring entry `service=raindrop.io` with no `UserName`;
+`cce-browser --raindrop-plan` is a read-only dry run against the real account,
+and `CCE_RAINDROP_API` points everything at a stand-in for testing.
+
## `cce://` pages
`CceProtocol` registers the `cce` scheme with Servo's `ProtocolRegistry`, so
@@ -606,7 +620,7 @@ must never carry an index across a lock boundary.
in `handle_focus_change` — so edits made in **cce-system-interface's Browser page**
(`../cce-system-interface/src/pages/browser.rs`, which owns the writing side) apply on
the next switch back. Keep the key names in `settings.rs` and that page in sync;
-`external-browser` is currently read here with no UI writing it.
+`external-browser` and `raindrop` are currently read here with no UI writing them.
Servo persists per-profile state (cookie jar, auth cache, HSTS) only when given a
`config_dir` — without one every launch starts logged out of every site. It lives at
diff --git a/RAINDROP-SYNC.md b/RAINDROP-SYNC.md
index 88394f6..44becc2 100644
--- a/RAINDROP-SYNC.md
+++ b/RAINDROP-SYNC.md
@@ -1,10 +1,11 @@
# Bookmark sync with Raindrop.io
-Status: **phase 2 done** (2026-10-02). Phase 1 is the merge, the deletion
-guard, the sync state file and applying a plan locally (`src/raindrop/mod.rs`);
-phase 2 is the REST client, the keyring token and a read-only dry run
-(`src/raindrop/api.rs`, `cce-browser --raindrop-plan`). 25 unit tests, the
-client's against a stand-in server. Nothing syncs on its own yet — phase 3.
+Status: **phase 3 done** (2026-10-02) — the browser syncs on its own when
+`browser.raindrop` is on. Phase 1 is the merge (`src/raindrop/mod.rs`), phase 2
+the REST client, keyring token and `cce-browser --raindrop-plan` dry run
+(`src/raindrop/api.rs`), phase 3 the worker and the status line
+(`src/raindrop/sync.rs`). 30 unit tests, plus one for the page links; phase 3 was also run end to end in a
+shadow session against a stand-in Raindrop and a throwaway keyring.
## Decisions
@@ -103,13 +104,50 @@ offered to a login form. A locked keyring is an error, never a prompt.
`bookmarks.tsv` and the base, prints it, and changes nothing. It runs ahead of
the single-instance hand-off, so it works while the browser is open.
-## Phase 3 — wiring
-
-The worker, the debounce, `apply_local` against `pages::Bookmarks` under its
-lock, `save_base` last. A one-line status on `cce://bookmarks` ("synced 3m
-ago", or why not, including a refused plan with a way to force it). Setting
-`browser.raindrop` (off by default); its writing side belongs to
-cce-system-interface's Browser page — keep the key names in sync.
+## Phase 3 — the worker (done)
+
+`browser.raindrop` (KDL, `browser { raindrop (bool)true }`, off by default) is
+read at launch and on every focus like the other settings; the worker thread
+`cce-raindrop` starts the first time it is on and idles while it is off. Its
+writing side belongs to cce-system-interface's Browser page and does not exist
+yet — like `external-browser`, it is edited by hand for now. Choices:
+
+- **It polls the bookmarks every 2 s instead of being told about edits.** A
+ bookmark changes from the star, Ctrl+D, the bookmarks menu and the
+ `cce://bookmarks` page; comparing the in-memory rows (no I/O) catches all of
+ them with no hook in each. A change syncs once it has held still for one
+ poll, so a burst of edits is one pass. A full pass every 10 minutes brings
+ Raindrop's side; the page's "sync now" asks for one at once.
+- **A pass's local half is one locked edit** (`Bookmarks::edit_rows`), so no
+ star or remove can land in the middle of it, and it is skipped entirely when
+ the plan changes nothing here — an idle pass never rewrites the file. The
+ worker re-reads the rows after its own pass, so its edits are not mistaken
+ for the person's.
+- **Titles are flattened where they arrive** (`api::parse_item`): a tab or a
+ line break in a Raindrop title would otherwise be flattened by the TSV store
+ and read as "renamed in Raindrop" on every pass.
+- **A skipped link edit leaves the base** (`settle_base`): otherwise its pair
+ would point at a URL that is not here and the next pass would trash
+ Raindrop's copy.
+- **The status line lives on `cce://bookmarks`**, set by the worker through
+ `Bookmarks::set_sync_note`: what the last pass did and when, with "sync now".
+ A refused pass shows why and a **"sync anyway"** link carrying a random code
+ that `cce://bookmarks/sync-force` checks — a web page linking to `cce://`
+ cannot force a mass deletion, since it cannot read the page to learn the
+ code. The code holds for as long as passes keep being refused (a periodic
+ pass, a reload); a fresh one each time made the link on screen stale, which
+ is how the end-to-end run found it. After a forced pass the code is gone, so
+ reloading its URL forces nothing.
+- The page is static: the line is as of the page's load, and reloading
+ `cce://bookmarks/sync` asks for another (harmless) pass.
+
+**Testing without an account:** `CCE_RAINDROP_API=<base url>` points the
+client at a stand-in (it logs a warning when it does). The end-to-end run used
+a ~60-line Python stand-in for Unsorted (GET/POST/PUT/DELETE, held in memory),
+a throwaway keyring holding a fake token (the autofill section of CLAUDE.md has
+the recipe and its two traps), and checked: first pass imports and creates; a
+local remove trashes; a rename and a save on the "phone" arrive; an emptied
+collection is refused with the bookmarks untouched; "sync anyway" runs it.
## Phase 4 — favorites (optional)
diff --git a/src/main.rs b/src/main.rs
index c8d17d1..14ca807 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -592,6 +592,10 @@ struct BrowserApp {
accounts: accounts::Accounts,
/// Hosts the person said never to offer saving on.
never_save: accounts::NeverSave,
+ /// The `browser.raindrop` setting, shared live with the sync worker,
+ /// which is started the first time it is on (RAINDROP-SYNC.md).
+ raindrop_on: std::sync::Arc<std::sync::atomic::AtomicBool>,
+ raindrop_started: bool,
/// A new sign-in waiting for Save / Never / Not now.
#[cfg(feature = "wpe")]
save_offer: Option<SaveOffer>,
@@ -1814,6 +1818,11 @@ impl BrowserApp {
self.ac_menu = None;
}
}
+ self.raindrop_on.store(new.raindrop, std::sync::atomic::Ordering::SeqCst);
+ if new.raindrop && !self.raindrop_started {
+ raindrop::sync::spawn(self.host.bookmarks(), self.raindrop_on.clone());
+ self.raindrop_started = true;
+ }
self.host.set_history_enabled(new.history);
self.host.set_color_scheme_dark(new.color_scheme.is_dark());
self.host.set_force_dark(new.color_scheme.forces_dark());
@@ -2440,6 +2449,11 @@ impl Application for BrowserApp {
#[cfg(feature = "wpe")]
host.set_accounts_enabled(settings.accounts);
let accounts = accounts::Accounts::spawn(sender.clone());
+ let raindrop_on = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(settings.raindrop));
+ let raindrop_started = settings.raindrop;
+ if raindrop_started {
+ raindrop::sync::spawn(host.bookmarks(), raindrop_on.clone());
+ }
let favorites = host.favorites();
let favs = favorites.snapshot();
let bookmarks = host.bookmarks();
@@ -2475,6 +2489,8 @@ impl Application for BrowserApp {
scroll_sent: (0.0, 0.0),
accounts,
never_save: accounts::NeverSave::load(),
+ raindrop_on,
+ raindrop_started,
#[cfg(feature = "wpe")]
save_offer: None,
#[cfg(feature = "wpe")]
diff --git a/src/pages.rs b/src/pages.rs
index 0947118..ee5f2c2 100644
--- a/src/pages.rs
+++ b/src/pages.rs
@@ -193,12 +193,96 @@ impl History {
pub struct Bookmarks {
entries: Mutex<Vec<Entry>>,
path: PathBuf,
+ /// The Raindrop sync's last word, for the page to show. `None` while the
+ /// sync is off.
+ sync_note: Mutex<Option<SyncNote>>,
+ /// Raised by the page's sync links for the sync worker to pick up — the
+ /// handler cannot do network work itself. 1 = sync now, 2 = run the
+ /// refused pass anyway.
+ sync_request: std::sync::atomic::AtomicU8,
}
+/// What the Raindrop sync last said, shown on `cce://bookmarks`.
+#[derive(Clone, Debug, Default)]
+pub struct SyncNote {
+ pub text: String,
+ /// When it was said, for "3m ago".
+ pub at: u64,
+ /// Set when a pass was refused: the code the "sync anyway" link must
+ /// carry. Random per refusal, so a page that links to `cce://` cannot
+ /// force a mass deletion — it cannot read this page to learn the code.
+ pub force_code: Option<u64>,
+}
+
+pub const SYNC_NOW: u8 = 1;
+pub const SYNC_FORCE: u8 = 2;
+
impl Bookmarks {
pub fn load() -> Self {
- let path = state_dir().join("bookmarks.tsv");
- Self { entries: Mutex::new(read_tsv(&path)), path }
+ Self::at(state_dir().join("bookmarks.tsv"))
+ }
+
+ pub(crate) fn at(path: PathBuf) -> Self {
+ Self {
+ entries: Mutex::new(read_tsv(&path)),
+ path,
+ sync_note: Mutex::new(None),
+ sync_request: std::sync::atomic::AtomicU8::new(0),
+ }
+ }
+
+ /// Edit every row at once, under the lock, and write the file — the
+ /// Raindrop sync's way in, so a whole pass lands as one change that no
+ /// star or remove can interleave with. Fields are sanitized on the way
+ /// back: a title from elsewhere can hold a tab or a newline, and this
+ /// format has no escaping.
+ pub fn edit_rows<R>(&self, f: impl FnOnce(&mut Vec<(u64, String, String)>) -> R) -> R {
+ let mut entries = self.entries.lock().unwrap();
+ let mut rows: Vec<_> =
+ entries.iter().map(|e| (e.ts, e.url.clone(), e.title.clone())).collect();
+ let out = f(&mut rows);
+ *entries = rows
+ .into_iter()
+ .map(|(ts, url, title)| Entry { ts, url: sanitize(&url), title: sanitize(&title) })
+ .collect();
+ write_tsv(&self.path, &entries);
+ out
+ }
+
+ pub fn set_sync_note(&self, note: Option<SyncNote>) {
+ *self.sync_note.lock().unwrap() = note;
+ }
+
+ /// The "sync anyway" code on the page right now, if a refusal is showing.
+ pub fn sync_force_code(&self) -> Option<u64> {
+ self.sync_note.lock().unwrap().as_ref().and_then(|n| n.force_code)
+ }
+
+ /// The page's pending sync request, cleared as it is read.
+ pub fn take_sync_request(&self) -> u8 {
+ self.sync_request.swap(0, std::sync::atomic::Ordering::SeqCst)
+ }
+
+ /// The sync status line and its links, or nothing while the sync is off.
+ fn sync_html(&self) -> String {
+ let Some(note) = self.sync_note.lock().unwrap().clone() else {
+ return String::new();
+ };
+ let ago = now().saturating_sub(note.at);
+ let ago = match ago {
+ 0..=59 => "just now".to_string(),
+ 60..=3599 => format!("{}m ago", ago / 60),
+ _ => format!("{}h ago", ago / 3600),
+ };
+ let force = note
+ .force_code
+ .map(|c| format!(" <a class=rm href=\"cce://bookmarks/sync-force?code={c}\">sync anyway</a>"))
+ .unwrap_or_default();
+ format!(
+ "<div class=e><span class=w></span><span class=u>Raindrop: {} · {ago}</span>\
+ <a class=rm href=\"cce://bookmarks/sync\">sync now</a>{force}</div>\n",
+ html_escape(¬e.text)
+ )
}
pub fn contains(&self, url: &str) -> bool {
@@ -296,6 +380,8 @@ impl Bookmarks {
} else {
rows
};
+ drop(entries);
+ let body = format!("{}{body}", self.sync_html());
page("Bookmarks", &meta, &body, "")
}
}
@@ -539,6 +625,21 @@ impl CceProtocol {
}
Some(self.bookmarks.html(&self.favorites))
}
+ "bookmarks/sync" => {
+ self.bookmarks.sync_request.store(SYNC_NOW, std::sync::atomic::Ordering::SeqCst);
+ Some(self.bookmarks.html(&self.favorites))
+ }
+ // Only with the code the refusal put on this page: forcing a pass
+ // the guard refused is how a mass deletion happens on purpose, and
+ // it must not happen because some web page linked here.
+ "bookmarks/sync-force" => {
+ let expected = self.bookmarks.sync_note.lock().unwrap().as_ref().and_then(|n| n.force_code);
+ let given = param("code").and_then(|c| c.parse::<u64>().ok());
+ if expected.is_some() && given == expected {
+ self.bookmarks.sync_request.store(SYNC_FORCE, std::sync::atomic::Ordering::SeqCst);
+ }
+ Some(self.bookmarks.html(&self.favorites))
+ }
"favorites" => Some(self.favorites.html()),
// Adding lands on the favorites page so the new pill's place in
// the strip is visible right away. A bookmark promoted without a
@@ -590,6 +691,43 @@ impl CceProtocol {
mod tests {
use super::*;
+ #[test]
+ fn sync_links_raise_requests_and_force_needs_the_code() {
+ let dir = std::env::temp_dir().join(format!("cce-browser-sync-{}", std::process::id()));
+ let _ = std::fs::remove_dir_all(&dir);
+ let proto = CceProtocol {
+ history: Arc::new(History { entries: Mutex::new(Vec::new()), path: dir.join("history.tsv") }),
+ bookmarks: Arc::new(Bookmarks::at(dir.join("bookmarks.tsv"))),
+ favorites: Arc::new(Favorites { entries: Mutex::new(Vec::new()), path: dir.join("favorites.tsv") }),
+ downloads: Arc::new(crate::downloads::Downloads::default()),
+ clear_cookies: Arc::new(std::sync::atomic::AtomicBool::new(false)),
+ };
+ let b = &proto.bookmarks;
+ // Off: no status line at all.
+ assert!(!proto.route("cce://bookmarks").unwrap().contains("Raindrop:"));
+
+ b.set_sync_note(Some(SyncNote { text: "synced".into(), at: now(), force_code: None }));
+ assert!(proto.route("cce://bookmarks").unwrap().contains("Raindrop: synced · just now"));
+ proto.route("cce://bookmarks/sync");
+ assert_eq!(b.take_sync_request(), SYNC_NOW);
+ assert_eq!(b.take_sync_request(), 0, "a request is taken once");
+
+ // Forcing needs a refusal, and its code.
+ proto.route("cce://bookmarks/sync-force?code=7");
+ assert_eq!(b.take_sync_request(), 0, "nothing was refused");
+ b.set_sync_note(Some(SyncNote { text: "refused".into(), at: now(), force_code: Some(42) }));
+ assert!(proto.route("cce://bookmarks").unwrap().contains("sync-force?code=42"));
+ proto.route("cce://bookmarks/sync-force?code=7");
+ assert_eq!(b.take_sync_request(), 0, "the wrong code forces nothing");
+ proto.route("cce://bookmarks/sync-force?code=42");
+ assert_eq!(b.take_sync_request(), SYNC_FORCE);
+
+ // A pass's edit sanitizes what it writes.
+ b.edit_rows(|rows| rows.push((1, "https://a.test/".into(), "two\nlines\there".into())));
+ assert_eq!(b.rows(), vec![(1, "https://a.test/".to_string(), "two lines here".to_string())]);
+ let _ = std::fs::remove_dir_all(&dir);
+ }
+
/// A favorites store in a scratch directory of this TEST's own.
///
/// Named per test rather than shared. The tests run in parallel threads
@@ -651,7 +789,7 @@ mod tests {
fn bookmarks_list_newest_first_with_labelled_entries() {
let dir = std::env::temp_dir().join(format!("cce-browser-bm-{}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
- let b = Bookmarks { entries: Mutex::new(Vec::new()), path: dir.join("bookmarks.tsv") };
+ let b = Bookmarks::at(dir.join("bookmarks.tsv"));
b.toggle("https://www.first.example/a", "First");
b.toggle("https://second.example/b", "");
let seen: Vec<(String, String)> =
diff --git a/src/raindrop/api.rs b/src/raindrop/api.rs
index 57f2a3f..cd4a912 100644
--- a/src/raindrop/api.rs
+++ b/src/raindrop/api.rs
@@ -64,8 +64,17 @@ pub struct Client {
}
impl Client {
+ /// The real Raindrop — or, when `CCE_RAINDROP_API` is set, a stand-in at
+ /// that base URL, which is how the whole browser is tested end to end
+ /// without writing to an account.
pub fn new(token: Secret) -> Self {
- Self::with_base(token, BASE)
+ match std::env::var("CCE_RAINDROP_API") {
+ Ok(base) if !base.is_empty() => {
+ log::warn!("raindrop: using the stand-in API at {base}");
+ Self::with_base(token, &base)
+ }
+ _ => Self::with_base(token, BASE),
+ }
}
/// Against another server — the tests' stand-in.
@@ -184,10 +193,14 @@ impl Client {
fn parse_item(v: &serde_json::Value) -> Result<Remote, ApiError> {
let id = v["_id"].as_u64().ok_or_else(|| ApiError::Parse("an item has no _id".into()))?;
let link = v["link"].as_str().ok_or_else(|| ApiError::Parse(format!("item {id} has no link")))?;
+ // Tabs and line breaks become spaces here, at the source: the local
+ // store has no escaping and flattens them, and a title that differed
+ // only in that way would read as renamed in Raindrop on every pass.
+ let flat = |s: &str| s.replace(['\t', '\n', '\r'], " ");
Ok(Remote {
id,
- link: link.to_string(),
- title: v["title"].as_str().unwrap_or_default().to_string(),
+ link: flat(link),
+ title: flat(v["title"].as_str().unwrap_or_default()),
created: v["created"].as_str().and_then(iso8601_secs).unwrap_or(0),
})
}
@@ -302,14 +315,14 @@ pub fn describe(plan: &Plan, local: &[Local], remote: &[Remote], base: &[Synced]
}
#[cfg(test)]
-mod tests {
+pub(super) mod tests {
use super::*;
use std::io::{BufRead, BufReader, Read, Write};
use std::sync::{Arc, Mutex};
/// A stand-in Raindrop: answers each request with the next scripted
/// `(status, extra headers, body)` and records `(request line, body)`.
- fn server(script: Vec<(u16, &'static str, String)>) -> (String, Arc<Mutex<Vec<(String, String)>>>) {
+ pub(in crate::raindrop) fn server(script: Vec<(u16, &'static str, String)>) -> (String, Arc<Mutex<Vec<(String, String)>>>) {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}", listener.local_addr().unwrap());
let seen = Arc::new(Mutex::new(Vec::new()));
@@ -354,7 +367,7 @@ mod tests {
(base, seen)
}
- fn items(range: std::ops::Range<u64>, count: usize) -> String {
+ pub(in crate::raindrop) fn items(range: std::ops::Range<u64>, count: usize) -> String {
let items: Vec<_> = range
.map(|i| serde_json::json!({
"_id": i, "link": format!("https://{i}.test/"), "title": format!("t{i}"),
@@ -447,6 +460,12 @@ mod tests {
assert_eq!(applied.errors.len(), 2);
}
+ #[test]
+ fn titles_are_flattened_where_they_arrive() {
+ let r = parse_item(&serde_json::json!({"_id": 1, "link": "https://a.test/", "title": "two\nlines\tand tab"})).unwrap();
+ assert_eq!(r.title, "two lines and tab");
+ }
+
#[test]
fn timestamps_parse() {
assert_eq!(iso8601_secs("1970-01-01T00:00:00Z"), Some(0));
diff --git a/src/raindrop/mod.rs b/src/raindrop/mod.rs
index 7da15ff..7443ecd 100644
--- a/src/raindrop/mod.rs
+++ b/src/raindrop/mod.rs
@@ -25,10 +25,8 @@
//! * **Raindrop's duplicates are left alone.** Locally a URL is a key; a second
//! Raindrop entry for a link already here is neither imported nor removed.
-// Not wired into the browser until phase 3; the tests exercise all of it.
-#![allow(dead_code)]
-
pub mod api;
+pub mod sync;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
@@ -292,18 +290,30 @@ pub fn guard(plan: &Plan, local_len: usize, remote_len: usize, base_len: usize)
None
}
+/// What [`apply_local`] left undone.
+#[derive(Debug, Default, PartialEq)]
+pub struct Skipped {
+ pub count: usize,
+ /// The *new* URLs of link edits not applied. Their pairs must leave the
+ /// base: it would pair the id with a URL that is not here, and the next
+ /// pass would read that as "deleted here" and trash the Raindrop copy.
+ /// Dropped from the base, the two simply re-pair (or both survive).
+ pub relinks: Vec<String>,
+}
+
/// Apply a plan's local half to the bookmarks as they are **now**.
///
/// The plan was made from a `snapshot`, and the person may have bookmarked or
/// removed something while Raindrop was answering. An operation on a URL whose
/// entry is not what the snapshot had is skipped — the next pass sees the new
/// state and plans again — so a local edit is never overwritten by a stale
-/// plan. Returns how many were skipped.
-pub fn apply_local(current: &mut Vec<Local>, snapshot: &[Local], plan: &Plan) -> usize {
+/// plan.
+pub fn apply_local(current: &mut Vec<Local>, snapshot: &[Local], plan: &Plan) -> Skipped {
let unchanged = |current: &Vec<Local>, url: &str| {
current.iter().find(|l| l.url == url) == snapshot.iter().find(|l| l.url == url)
};
let mut skipped = 0;
+ let mut relinks = Vec::new();
for (old, new) in &plan.relink_local {
if unchanged(current, old) && !current.iter().any(|l| l.url == *new) {
if let Some(l) = current.iter_mut().find(|l| l.url == *old) {
@@ -311,12 +321,14 @@ pub fn apply_local(current: &mut Vec<Local>, snapshot: &[Local], plan: &Plan) ->
}
} else {
skipped += 1;
+ relinks.push(new.clone());
}
}
for (url, title) in &plan.rename_local {
// Renames address the post-relink URL; a relinked entry was checked
// against the snapshot by its old one above.
- let relinked = plan.relink_local.iter().any(|(_, n)| n == url);
+ let relinked =
+ plan.relink_local.iter().any(|(_, n)| n == url) && !relinks.contains(url);
if relinked || unchanged(current, url) {
if let Some(l) = current.iter_mut().find(|l| l.url == *url) {
l.title = title.clone();
@@ -341,7 +353,15 @@ pub fn apply_local(current: &mut Vec<Local>, snapshot: &[Local], plan: &Plan) ->
}
// Local order is bookmarking time; an import lands where it was made.
current.sort_by_key(|l| l.ts);
- skipped
+ Skipped { count: skipped, relinks }
+}
+
+/// The base to save after a pass: `base_after`, minus the pairs of link edits
+/// that were skipped here (see [`Skipped::relinks`]).
+pub fn settle_base(plan: &Plan, prior: &[Synced], applied: &api::Applied, skipped: &Skipped) -> Vec<Synced> {
+ let mut base = plan.base_after(prior, applied);
+ base.retain(|b| !skipped.relinks.contains(&b.url));
+ base
}
/// `cce-browser --raindrop-plan`: fetch Unsorted, plan a pass against the
@@ -445,7 +465,7 @@ mod tests {
created.push((c.url.clone(), id));
}
let mut local_now = local.to_vec();
- assert_eq!(apply_local(&mut local_now, local, &p), 0);
+ assert_eq!(apply_local(&mut local_now, local, &p).count, 0);
let applied = api::Applied { created, ..Default::default() };
(local_now, remote, p.base_after(base, &applied))
}
@@ -625,12 +645,28 @@ mod tests {
};
// Meanwhile: a.test re-bookmarked with a new title, c.test bookmarked.
let mut current = vec![l("https://a.test/", "A again", 5), l("https://b.test/", "B", 2), l("https://c.test/", "mine", 6)];
- assert_eq!(apply_local(&mut current, &snapshot, &p), 2);
+ assert_eq!(apply_local(&mut current, &snapshot, &p).count, 2);
assert!(current.iter().any(|x| x.url == "https://a.test/" && x.title == "A again"));
assert!(current.iter().any(|x| x.url == "https://b.test/" && x.title == "B2"));
assert!(current.iter().any(|x| x.url == "https://c.test/" && x.title == "mine"));
}
+ #[test]
+ fn a_skipped_link_edit_cannot_become_a_deletion() {
+ let prior = [s(1, "http://old.test/", "P")];
+ let snapshot = vec![l("http://old.test/", "P", 1)];
+ let remote = [r(1, "https://new.test/", "P")];
+ let p = plan(&snapshot, &remote, &prior).unwrap();
+ // Mid-pass the person re-bookmarked the old link with a new title.
+ let mut current = vec![l("http://old.test/", "P again", 9)];
+ let skipped = apply_local(&mut current, &snapshot, &p);
+ assert_eq!(skipped.relinks, vec!["https://new.test/".to_string()]);
+ let base = settle_base(&p, &prior, &api::Applied::default(), &skipped);
+ let next = plan(¤t, &remote, &base).unwrap();
+ assert!(next.trash_remote.is_empty(), "Raindrop's copy is not trashed");
+ assert!(next.delete_local.is_empty(), "and neither is the local one");
+ }
+
#[test]
fn the_base_round_trips_and_drops_damage() {
let dir = std::env::temp_dir().join(format!("cce-raindrop-{}", std::process::id()));
diff --git a/src/raindrop/sync.rs b/src/raindrop/sync.rs
new file mode 100644
index 0000000..d407450
--- /dev/null
+++ b/src/raindrop/sync.rs
@@ -0,0 +1,310 @@
+//! The sync worker — phase 3: when a pass runs, and what it says.
+//!
+//! One thread, `cce-raindrop`, for the life of the browser once the setting
+//! has been on. It polls the in-memory bookmarks every [`POLL`] instead of
+//! being told about edits: a bookmark changes from the star, Ctrl+D, the
+//! bookmarks menu and the `cce://bookmarks` page, and comparing the rows
+//! (no I/O, a lock held for a copy) catches all of them without a hook in
+//! each. A change syncs once the rows have held still for one poll, so a
+//! burst of edits is one pass; Raindrop's own changes arrive with the pass
+//! every [`FULL`]; the page's "sync now" asks for one at once.
+//!
+//! A pass is [`run_pass`]: fetch, plan, apply Raindrop's half, apply the local
+//! half through `Bookmarks::edit_rows` (one locked edit — no star can land in
+//! the middle of it), and save the base last. Its outcome becomes the status
+//! line on `cce://bookmarks`.
+
+use std::path::Path;
+use std::sync::atomic::{AtomicBool, Ordering};
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+use super::{api, apply_local, load_base, plan, save_base, settle_base, Local};
+use crate::pages::{Bookmarks, SyncNote, SYNC_FORCE};
+
+/// How often the rows are compared, and the stillness a change waits for.
+const POLL: Duration = Duration::from_secs(2);
+/// A full pass this often while nothing local changes: Raindrop's side.
+const FULL: Duration = Duration::from_secs(600);
+
+/// What one pass did.
+#[derive(Debug, Default, PartialEq)]
+pub struct Summary {
+ pub added_here: usize,
+ pub removed_here: usize,
+ pub changed_here: usize,
+ pub added_there: usize,
+ pub trashed_there: usize,
+ pub renamed_there: usize,
+ /// Local operations left for the next pass (edited mid-pass).
+ pub deferred: usize,
+ /// Raindrop calls that failed; each is retried next pass.
+ pub errors: Vec<String>,
+}
+
+impl Summary {
+ /// The status line: what moved, or "in sync".
+ pub fn text(&self) -> String {
+ let mut parts = Vec::new();
+ let mut say = |n: usize, what: &str| {
+ if n > 0 {
+ parts.push(format!("{n} {what}"));
+ }
+ };
+ say(self.added_here, "added here");
+ say(self.removed_here, "removed here");
+ say(self.changed_here, "updated here");
+ say(self.added_there, "added to Raindrop");
+ say(self.trashed_there, "moved to Raindrop's trash");
+ say(self.renamed_there, "renamed in Raindrop");
+ let mut s = if parts.is_empty() { "in sync".to_string() } else { format!("synced — {}", parts.join(", ")) };
+ if !self.errors.is_empty() {
+ s.push_str(&format!(" ({} failed, retrying next pass)", self.errors.len()));
+ }
+ s
+ }
+}
+
+#[derive(Debug, PartialEq)]
+pub enum Outcome {
+ Synced(Summary),
+ /// The guard refused the pass; nothing was changed on either side.
+ Refused(String),
+}
+
+fn locals(rows: Vec<(u64, String, String)>) -> Vec<Local> {
+ rows.into_iter().map(|(ts, url, title)| Local { url, title, ts }).collect()
+}
+
+/// One pass against `client`'s Unsorted. `force` runs a plan the guard
+/// refused — only ever from the page's "sync anyway", after a person has read
+/// why it was refused.
+pub fn run_pass(
+ client: &api::Client,
+ bookmarks: &Bookmarks,
+ base_path: &Path,
+ force: bool,
+) -> Result<Outcome, String> {
+ let remote = client.fetch(api::UNSORTED).map_err(|e| e.to_string())?;
+ let snapshot = locals(bookmarks.rows());
+ let base = load_base(base_path);
+ let plan = match plan(&snapshot, &remote, &base) {
+ Ok(p) => p,
+ Err(refused) if force => {
+ log::warn!("raindrop: running a refused pass on request ({})", refused.reason);
+ refused.plan
+ }
+ Err(refused) => return Ok(Outcome::Refused(refused.reason)),
+ };
+ let applied = api::apply_remote(client, api::UNSORTED, &plan).map_err(|e| e.to_string())?;
+ let touches_local = !(plan.relink_local.is_empty()
+ && plan.rename_local.is_empty()
+ && plan.delete_local.is_empty()
+ && plan.add_local.is_empty());
+ // Only rewrite the file when there is something to write: a pass every
+ // ten minutes that changes nothing must not touch the disk.
+ let skipped = if touches_local {
+ bookmarks.edit_rows(|rows| {
+ let mut current = locals(std::mem::take(rows));
+ let skipped = apply_local(&mut current, &snapshot, &plan);
+ *rows = current.into_iter().map(|l| (l.ts, l.url, l.title)).collect();
+ skipped
+ })
+ } else {
+ super::Skipped::default()
+ };
+ // Last: a pass that dies before this point leaves the old base, and the
+ // next pass redoes the work rather than misreading it.
+ let new_base = settle_base(&plan, &base, &applied, &skipped);
+ if new_base != base {
+ save_base(base_path, &new_base).map_err(|e| format!("could not save the sync state: {e}"))?;
+ }
+ Ok(Outcome::Synced(Summary {
+ added_here: plan.add_local.len(),
+ removed_here: plan.delete_local.len(),
+ changed_here: plan.rename_local.len() + plan.relink_local.len(),
+ added_there: applied.created.len(),
+ trashed_there: plan.trash_remote.len() - applied.failed_trash.len(),
+ renamed_there: plan.rename_remote.len() - applied.failed_renames.len(),
+ deferred: skipped.count,
+ errors: applied.errors,
+ }))
+}
+
+/// A code for the "sync anyway" link. Not cryptographic, and it needs not be:
+/// it only has to be something a web page cannot know, and the page that
+/// shows it is the only place it is written.
+fn force_code() -> u64 {
+ use std::hash::{BuildHasher, Hasher};
+ let mut h = std::collections::hash_map::RandomState::new().build_hasher();
+ h.write_u64(super::unix_now());
+ h.finish()
+}
+
+/// One pass, start to status line: the token, the client, `run_pass`.
+fn pass(bookmarks: &Bookmarks, force: bool) {
+ let note = |text: String, force_code: Option<u64>| {
+ bookmarks.set_sync_note(Some(SyncNote { text, at: super::unix_now(), force_code }));
+ };
+ let token = match crate::accounts::raindrop_token() {
+ Ok(t) => t,
+ Err(e) => {
+ log::warn!("raindrop: {e}");
+ note(e, None);
+ return;
+ }
+ };
+ let client = api::Client::new(token);
+ match run_pass(&client, bookmarks, &super::state_path(), force) {
+ Ok(Outcome::Synced(s)) => {
+ for e in &s.errors {
+ log::warn!("raindrop: {e}");
+ }
+ log::info!("raindrop: {}", s.text());
+ note(s.text(), None);
+ }
+ Ok(Outcome::Refused(reason)) => {
+ log::warn!("raindrop: pass refused: {reason}");
+ // The same code for as long as passes keep being refused: a page
+ // already showing "sync anyway" must stay able to use it when a
+ // later pass — the periodic one, a reload — refuses again.
+ let code = bookmarks.sync_force_code().unwrap_or_else(force_code);
+ note(format!("not synced: {reason}"), Some(code));
+ }
+ Err(e) => {
+ log::warn!("raindrop: {e}");
+ note(format!("not synced: {e}"), None);
+ }
+ }
+}
+
+/// Start the worker. `enabled` is the setting, live: off, the worker idles
+/// and the page shows no status line; on again, it syncs at once.
+pub fn spawn(bookmarks: Arc<Bookmarks>, enabled: Arc<AtomicBool>) {
+ std::thread::Builder::new()
+ .name("cce-raindrop".to_string())
+ .spawn(move || {
+ let mut last_rows = None;
+ let mut changed = false;
+ let mut next_full = Instant::now();
+ let mut was_on = false;
+ loop {
+ std::thread::sleep(POLL);
+ let on = enabled.load(Ordering::SeqCst);
+ if !on {
+ if was_on {
+ bookmarks.set_sync_note(None);
+ }
+ was_on = false;
+ continue;
+ }
+ if !was_on {
+ // Just turned on (or launched on): sync now.
+ was_on = true;
+ next_full = Instant::now();
+ }
+ let request = bookmarks.take_sync_request();
+ let rows = bookmarks.rows();
+ if last_rows.as_ref() != Some(&rows) {
+ // Changed since the last look: wait one poll for it to
+ // hold still, so a burst of edits is one pass.
+ changed = last_rows.is_some();
+ last_rows = Some(rows);
+ if request == 0 {
+ continue;
+ }
+ }
+ if request != 0 || changed || Instant::now() >= next_full {
+ pass(&bookmarks, request == SYNC_FORCE);
+ changed = false;
+ next_full = Instant::now() + FULL;
+ // The pass's own edits are not a local change to sync.
+ last_rows = Some(bookmarks.rows());
+ }
+ }
+ })
+ .expect("spawn the Raindrop sync worker");
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::accounts::Secret;
+ use api::tests::{items, server};
+
+ fn scratch(name: &str) -> std::path::PathBuf {
+ let dir = std::env::temp_dir().join(format!("cce-raindrop-sync-{name}-{}", std::process::id()));
+ let _ = std::fs::remove_dir_all(&dir);
+ std::fs::create_dir_all(&dir).unwrap();
+ dir
+ }
+
+ #[test]
+ fn a_first_pass_imports_creates_and_saves_the_base() {
+ let dir = scratch("first");
+ let bookmarks = Bookmarks::at(dir.join("bookmarks.tsv"));
+ bookmarks.toggle("https://local.test/", "Mine");
+ // Raindrop: 1.test and 2.test; then the create for local.test.
+ let (base, seen) = server(vec![
+ (200, "", items(1..3, 2)),
+ (200, "", r#"{"result":true,"item":{"_id":90}}"#.into()),
+ ]);
+ let client = api::Client::with_base(Secret::from("t".to_string()), &base);
+ let base_path = dir.join("raindrop-sync.tsv");
+ let out = run_pass(&client, &bookmarks, &base_path, false).unwrap();
+ let Outcome::Synced(s) = out else { panic!("refused") };
+ assert_eq!((s.added_here, s.added_there), (2, 1));
+ assert_eq!(s.text(), "synced — 2 added here, 1 added to Raindrop");
+ let urls: Vec<_> = bookmarks.rows().into_iter().map(|r| r.1).collect();
+ assert_eq!(urls.len(), 3);
+ let saved = load_base(&base_path);
+ assert_eq!(saved.len(), 3, "both imports and the create are paired");
+ assert!(saved.iter().any(|b| b.id == 90 && b.url == "https://local.test/"));
+ assert_eq!(seen.lock().unwrap().len(), 2);
+ let _ = std::fs::remove_dir_all(dir);
+ }
+
+ #[test]
+ fn a_refused_pass_changes_nothing_until_forced() {
+ let dir = scratch("refused");
+ let bookmarks = Bookmarks::at(dir.join("bookmarks.tsv"));
+ let base_path = dir.join("raindrop-sync.tsv");
+ // Synced before: three bookmarks, all gone here now.
+ save_base(&base_path, &(1..4).map(|i| super::super::Synced {
+ id: i, url: format!("https://{i}.test/"), title: format!("t{i}"),
+ }).collect::<Vec<_>>()).unwrap();
+ let (base, seen) = server(vec![
+ (200, "", items(1..4, 3)),
+ (200, "", items(1..4, 3)),
+ (200, "", "{}".into()),
+ (200, "", "{}".into()),
+ (200, "", "{}".into()),
+ ]);
+ let client = api::Client::with_base(Secret::from("t".to_string()), &base);
+ let out = run_pass(&client, &bookmarks, &base_path, false).unwrap();
+ assert!(matches!(out, Outcome::Refused(ref r) if r.contains("every bookmark in Raindrop")), "{out:?}");
+ assert_eq!(seen.lock().unwrap().len(), 1, "only the fetch went out");
+ assert_eq!(load_base(&base_path).len(), 3, "the base is untouched");
+
+ let Outcome::Synced(s) = run_pass(&client, &bookmarks, &base_path, true).unwrap() else {
+ panic!("forced pass refused")
+ };
+ assert_eq!(s.trashed_there, 3);
+ assert!(load_base(&base_path).is_empty());
+ let _ = std::fs::remove_dir_all(dir);
+ }
+
+ #[test]
+ fn an_idle_pass_does_not_touch_the_file() {
+ let dir = scratch("idle");
+ let path = dir.join("bookmarks.tsv");
+ let bookmarks = Bookmarks::at(path.clone());
+ let (base, _) = server(vec![(200, "", items(1..1, 0))]);
+ let client = api::Client::with_base(Secret::from("t".to_string()), &base);
+ let out = run_pass(&client, &bookmarks, &dir.join("raindrop-sync.tsv"), false).unwrap();
+ assert_eq!(out, Outcome::Synced(Summary::default()));
+ assert_eq!(Summary::default().text(), "in sync");
+ assert!(!path.exists(), "nothing to write, nothing written");
+ let _ = std::fs::remove_dir_all(dir);
+ }
+}
diff --git a/src/settings.rs b/src/settings.rs
index 0c5795d..cd2bcc5 100644
--- a/src/settings.rs
+++ b/src/settings.rs
@@ -92,6 +92,10 @@ pub struct Settings {
/// single switch for the whole feature: with it off the browser injects
/// no watcher script and never opens the keyring.
pub accounts: bool,
+ /// Sync bookmarks with Raindrop.io's Unsorted collection
+ /// (RAINDROP-SYNC.md). Off by default: it needs a token in the keyring,
+ /// and it writes to an account elsewhere.
+ pub raindrop: bool,
/// Window edge the utility bar floats against.
pub bar_position: BarPosition,
/// What pages are told to prefer.
@@ -109,6 +113,7 @@ impl Default for Settings {
download_dir: None,
history: true,
accounts: true,
+ raindrop: false,
bar_position: BarPosition::Top,
color_scheme: ColorScheme::Dark,
external_browser: None,
@@ -152,6 +157,7 @@ pub fn load() -> Settings {
download_dir,
history: b["history"].as_bool().unwrap_or(true),
accounts: b["accounts"].as_bool().unwrap_or(true),
+ raindrop: b["raindrop"].as_bool().unwrap_or(false),
bar_position: BarPosition::from_key(b["bar-position"].as_str().unwrap_or("top")),
color_scheme: ColorScheme::from_key(b["color-scheme"].as_str().unwrap_or("dark")),
external_browser: b["external-browser"]