web browser (Servo)
git clone https://git.lucas.co/cce-browser.git
RAINDROP-SYNC.md (9.8K)
1 # Bookmark sync with Raindrop.io
2
3 Status: **phase 3 done** (2026-10-02) — the browser syncs on its own when
4 `browser.raindrop` is on. Phase 1 is the merge (`src/raindrop/mod.rs`), phase 2
5 the REST client, keyring token and `cce-browser --raindrop-plan` dry run
6 (`src/raindrop/api.rs`), phase 3 the worker and the status line
7 (`src/raindrop/sync.rs`). 30 unit tests, plus one for the page links; phase 3 was also run end to end in a
8 shadow session against a stand-in Raindrop and a throwaway keyring.
9
10 ## Decisions
11
12 - **One Raindrop collection — Unsorted (id `-1`) — mirrors the browser's
13 bookmarks**, and new bookmarks from the browser land there. It is also where
14 Raindrop's phone app and extensions save by default, so a bookmark saved
15 anywhere shows up here. The cost, accepted: filing a bookmark into another
16 collection in Raindrop moves it out of the mirror, and the next pass removes
17 it here (not from Raindrop). Mirroring one collection and creating in another
18 was ruled out — every bookmark made here would read as deleted there.
19 - **Deleting here moves the Raindrop copy to its trash** (recoverable there).
20 - **Favorites stay local.** A dedicated Raindrop collection is the upgrade path
21 (phase 4), using its manual order for strip order.
22
23 ## Shape
24
25 A worker thread **inside the browser**, not a separate tool like
26 cce-keyring-sync: only the browser edits bookmarks locally, and it holds them
27 in memory (`pages::Bookmarks`, rewriting `bookmarks.tsv` on each edit), so a
28 second process writing that file would race it. It syncs at launch, a few
29 seconds after a local edit (debounced), and every ~10 minutes while running.
30 The menu and `cce://bookmarks` keep reading the local store and never wait on
31 the network.
32
33 ## The merge (phase 1, `src/raindrop.rs`)
34
35 `bookmarks.tsv` is unchanged. `raindrop-sync.tsv` holds the **base** — `id \t
36 local url \t title` per pair, as of the last successful pass — and `plan(local,
37 remote, base)` is a three-way merge:
38
39 | here | Raindrop | in base | action |
40 | --- | --- | --- | --- |
41 | ✓ | ✓ | ✓ | title: the side that moved off the base wins; both → Raindrop |
42 | ✓ | – | no | create in Raindrop |
43 | ✓ | – | ✓ | delete here (deleted there, or moved out of the collection) |
44 | – | ✓ | no | add here, at its `created` time |
45 | – | ✓ | ✓ | move to Raindrop's trash |
46
47 Rules, each with a test:
48
49 - **Identity is the Raindrop id.** URLs pair only what the base has never seen
50 (`pair_key`: scheme/host case and a trailing slash don't matter). A link
51 edited in Raindrop is followed locally (`relink_local`).
52 - **Only link and title are ever sent** — tags, notes and collections set
53 elsewhere cannot be clobbered (the cce-secrets attribute-wipe lesson).
54 - **A first run never deletes** (empty base: union only).
55 - **The guard refuses** more than 10 deletions on a side, emptying a side of 3
56 or more, or Raindrop answering empty when the base is not. The refused plan
57 is returned so a person can force it.
58 - **Local edits made during a pass survive it**: `apply_local` skips any
59 operation on an entry that changed since the snapshot the plan was made from.
60 - **A failed create retries next pass**, never reads as a deletion
61 (`base_after` only records ids Raindrop actually returned).
62 - **The base is written atomically** (temp file + rename): half a base would
63 read as half the bookmarks deleted.
64 - Raindrop's own duplicates are neither imported nor removed; `file:`
65 bookmarks stay local.
66 - A pass settles: re-planning a synced state is a no-op (tested end to end).
67
68 ## Phase 2 — the API client (done)
69
70 Checked against developer.raindrop.io on 2026-10-02: REST at
71 `api.raindrop.io/rest/v1`, `Authorization: Bearer <test token>` (from the
72 integration settings; test tokens do not expire), 120 requests/minute with
73 `429` past it, ISO 8601 timestamps. `GET /raindrops/-1` pages Unsorted 50 at a
74 time; `POST /raindrop` creates; `PUT /raindrop/{id}` is a **partial** update;
75 `DELETE /raindrop/{id}` moves to Trash — and is **permanent** on an item
76 already in Trash, so only ids just fetched from the live collection are ever
77 trashed. Choices:
78
79 - **A fetch must see every bookmark exactly once, or nothing is planned.**
80 Paging is by position within a sort, and **ties do not hold their order
81 between page requests**: bookmarks saved in one batch share a creation time
82 to the millisecond, and on the real 178-bookmark collection one read
83 returned 7 of them twice and 7 others never — with the *row* total still
84 equal to `count`, which is what the first version checked, so 5 bookmarks
85 were silently never imported (and a later pass could have read a missing
86 one as "deleted in Raindrop"). Found 2026-10-02, a day in. Now rows are kept
87 by id, the *distinct* count must equal `count`, and until it does the
88 collection is read again under another sort (`created`, `-created`,
89 `title`, …), each ordering the ties differently, and the reads combined.
90 A union that overshoots `count` means a deletion between reads, and is
91 refused like a read that never completes.
92 - **A failure on one item does not stop the rest**, and `base_after` records
93 what *happened*: a failed create stays out of the base (retried as new), a
94 failed trash keeps its pair (retried, not re-imported), a failed rename keeps
95 its old title (retried, not reversed). A `401` stops the pass at once.
96 - **One `429` is waited out** (until `X-RateLimit-Reset`, at most a minute).
97 - `reqwest` is now a plain dependency (it was Servo-only) — `blocking`, the
98 same build cce-map and cce-calendar use; JSON bodies are serialized by hand
99 rather than turning on its `json` feature, to keep it the same build.
100
101 **The token** lives in the keyring as an entry with no `UserName`, found by
102 `service=raindrop.io`:
103
104 ```sh
105 secret-tool store --label='Raindrop.io token' service raindrop.io
106 ```
107
108 cce-keyring-sync skips entries without `UserName`, so it stays on this machine
109 and never goes to 1Password; the account index skips it too, so it is never
110 offered to a login form. A locked keyring is an error, never a prompt.
111
112 **`cce-browser --raindrop-plan`** fetches Unsorted, plans a pass against
113 `bookmarks.tsv` and the base, prints it, and changes nothing. It runs ahead of
114 the single-instance hand-off, so it works while the browser is open.
115
116 ## Phase 3 — the worker (done)
117
118 `browser.raindrop` (KDL, `browser { raindrop (bool)true }`, off by default) is
119 read at launch and on every focus like the other settings; the worker thread
120 `cce-raindrop` starts the first time it is on and idles while it is off. Its
121 writing side is the "Sync Bookmarks with Raindrop" toggle on
122 cce-system-interface's Browser page (`src/pages/browser.rs` there) — keep the
123 key name in step with `settings.rs`. Choices:
124
125 - **It polls the bookmarks every 2 s instead of being told about edits.** A
126 bookmark changes from the star, Ctrl+D, the bookmarks menu and the
127 `cce://bookmarks` page; comparing the in-memory rows (no I/O) catches all of
128 them with no hook in each. A change syncs once it has held still for one
129 poll, so a burst of edits is one pass. A full pass every 10 minutes brings
130 Raindrop's side; the page's "sync now" asks for one at once.
131 - **A pass's local half is one locked edit** (`Bookmarks::edit_rows`), so no
132 star or remove can land in the middle of it, and it is skipped entirely when
133 the plan changes nothing here — an idle pass never rewrites the file. The
134 worker re-reads the rows after its own pass, so its edits are not mistaken
135 for the person's.
136 - **Titles are flattened where they arrive** (`api::parse_item`): a tab or a
137 line break in a Raindrop title would otherwise be flattened by the TSV store
138 and read as "renamed in Raindrop" on every pass.
139 - **A skipped link edit leaves the base** (`settle_base`): otherwise its pair
140 would point at a URL that is not here and the next pass would trash
141 Raindrop's copy.
142 - **The status line lives on `cce://bookmarks`**, set by the worker through
143 `Bookmarks::set_sync_note`: what the last pass did and when, with "sync now".
144 A refused pass shows why and a **"sync anyway"** link carrying a random code
145 that `cce://bookmarks/sync-force` checks — a web page linking to `cce://`
146 cannot force a mass deletion, since it cannot read the page to learn the
147 code. The code holds for as long as passes keep being refused (a periodic
148 pass, a reload); a fresh one each time made the link on screen stale, which
149 is how the end-to-end run found it. After a forced pass the code is gone, so
150 reloading its URL forces nothing.
151 - The page is static: the line is as of the page's load, and reloading
152 `cce://bookmarks/sync` asks for another (harmless) pass.
153 - **Every change is logged by name** in `raindrop-sync.log` beside the base:
154 `time \t what \t link \t title`, one line per bookmark added, removed or
155 renamed on either side, plus refusals, forced passes and failures. Local
156 lines are the difference the edit actually made and Raindrop lines the
157 calls that succeeded, so the log says what happened, not what was planned.
158 Passes that change nothing write nothing; past 512 KB it keeps its newer
159 half. Added after the first day's fetch bug, when "which two bookmarks came
160 back?" had no answer — the status line only counts, and the process log was
161 gone with the restart.
162
163 **Testing without an account:** `CCE_RAINDROP_API=<base url>` points the
164 client at a stand-in (it logs a warning when it does). The end-to-end run used
165 a ~60-line Python stand-in for Unsorted (GET/POST/PUT/DELETE, held in memory),
166 a throwaway keyring holding a fake token (the autofill section of CLAUDE.md has
167 the recipe and its two traps), and checked: first pass imports and creates; a
168 local remove trashes; a rename and a save on the "phone" arrive; an emptied
169 collection is refused with the bookmarks untouched; "sync anyway" runs it.
170
171 ## Phase 4 — favorites (optional)
172
173 A dedicated collection, ordered by Raindrop's manual sort.