git.lucas.co / cce-browser
web browser (Servo)
git clone https://git.lucas.co/cce-browser.git

RAINDROP-SYNC.md (9.8K)

  1 # Bookmark sync with Raindrop.io
  2 
  3 Status: **phase 3 done** (2026-10-02) — the browser syncs on its own when
  4 `browser.raindrop` is on. Phase 1 is the merge (`src/raindrop/mod.rs`), phase 2
  5 the REST client, keyring token and `cce-browser --raindrop-plan` dry run
  6 (`src/raindrop/api.rs`), phase 3 the worker and the status line
  7 (`src/raindrop/sync.rs`). 30 unit tests, plus one for the page links; phase 3 was also run end to end in a
  8 shadow session against a stand-in Raindrop and a throwaway keyring.
  9 
 10 ## Decisions
 11 
 12 - **One Raindrop collection — Unsorted (id `-1`) — mirrors the browser's
 13   bookmarks**, and new bookmarks from the browser land there. It is also where
 14   Raindrop's phone app and extensions save by default, so a bookmark saved
 15   anywhere shows up here. The cost, accepted: filing a bookmark into another
 16   collection in Raindrop moves it out of the mirror, and the next pass removes
 17   it here (not from Raindrop). Mirroring one collection and creating in another
 18   was ruled out — every bookmark made here would read as deleted there.
 19 - **Deleting here moves the Raindrop copy to its trash** (recoverable there).
 20 - **Favorites stay local.** A dedicated Raindrop collection is the upgrade path
 21   (phase 4), using its manual order for strip order.
 22 
 23 ## Shape
 24 
 25 A worker thread **inside the browser**, not a separate tool like
 26 cce-keyring-sync: only the browser edits bookmarks locally, and it holds them
 27 in memory (`pages::Bookmarks`, rewriting `bookmarks.tsv` on each edit), so a
 28 second process writing that file would race it. It syncs at launch, a few
 29 seconds after a local edit (debounced), and every ~10 minutes while running.
 30 The menu and `cce://bookmarks` keep reading the local store and never wait on
 31 the network.
 32 
 33 ## The merge (phase 1, `src/raindrop.rs`)
 34 
 35 `bookmarks.tsv` is unchanged. `raindrop-sync.tsv` holds the **base** — `id \t
 36 local url \t title` per pair, as of the last successful pass — and `plan(local,
 37 remote, base)` is a three-way merge:
 38 
 39 | here | Raindrop | in base | action |
 40 | --- | --- | --- | --- |
 41 | ✓ | ✓ | ✓ | title: the side that moved off the base wins; both → Raindrop |
 42 | ✓ | – | no | create in Raindrop |
 43 | ✓ | – | ✓ | delete here (deleted there, or moved out of the collection) |
 44 | – | ✓ | no | add here, at its `created` time |
 45 | – | ✓ | ✓ | move to Raindrop's trash |
 46 
 47 Rules, each with a test:
 48 
 49 - **Identity is the Raindrop id.** URLs pair only what the base has never seen
 50   (`pair_key`: scheme/host case and a trailing slash don't matter). A link
 51   edited in Raindrop is followed locally (`relink_local`).
 52 - **Only link and title are ever sent** — tags, notes and collections set
 53   elsewhere cannot be clobbered (the cce-secrets attribute-wipe lesson).
 54 - **A first run never deletes** (empty base: union only).
 55 - **The guard refuses** more than 10 deletions on a side, emptying a side of 3
 56   or more, or Raindrop answering empty when the base is not. The refused plan
 57   is returned so a person can force it.
 58 - **Local edits made during a pass survive it**: `apply_local` skips any
 59   operation on an entry that changed since the snapshot the plan was made from.
 60 - **A failed create retries next pass**, never reads as a deletion
 61   (`base_after` only records ids Raindrop actually returned).
 62 - **The base is written atomically** (temp file + rename): half a base would
 63   read as half the bookmarks deleted.
 64 - Raindrop's own duplicates are neither imported nor removed; `file:`
 65   bookmarks stay local.
 66 - A pass settles: re-planning a synced state is a no-op (tested end to end).
 67 
 68 ## Phase 2 — the API client (done)
 69 
 70 Checked against developer.raindrop.io on 2026-10-02: REST at
 71 `api.raindrop.io/rest/v1`, `Authorization: Bearer <test token>` (from the
 72 integration settings; test tokens do not expire), 120 requests/minute with
 73 `429` past it, ISO 8601 timestamps. `GET /raindrops/-1` pages Unsorted 50 at a
 74 time; `POST /raindrop` creates; `PUT /raindrop/{id}` is a **partial** update;
 75 `DELETE /raindrop/{id}` moves to Trash — and is **permanent** on an item
 76 already in Trash, so only ids just fetched from the live collection are ever
 77 trashed. Choices:
 78 
 79 - **A fetch must see every bookmark exactly once, or nothing is planned.**
 80   Paging is by position within a sort, and **ties do not hold their order
 81   between page requests**: bookmarks saved in one batch share a creation time
 82   to the millisecond, and on the real 178-bookmark collection one read
 83   returned 7 of them twice and 7 others never — with the *row* total still
 84   equal to `count`, which is what the first version checked, so 5 bookmarks
 85   were silently never imported (and a later pass could have read a missing
 86   one as "deleted in Raindrop"). Found 2026-10-02, a day in. Now rows are kept
 87   by id, the *distinct* count must equal `count`, and until it does the
 88   collection is read again under another sort (`created`, `-created`,
 89   `title`, …), each ordering the ties differently, and the reads combined.
 90   A union that overshoots `count` means a deletion between reads, and is
 91   refused like a read that never completes.
 92 - **A failure on one item does not stop the rest**, and `base_after` records
 93   what *happened*: a failed create stays out of the base (retried as new), a
 94   failed trash keeps its pair (retried, not re-imported), a failed rename keeps
 95   its old title (retried, not reversed). A `401` stops the pass at once.
 96 - **One `429` is waited out** (until `X-RateLimit-Reset`, at most a minute).
 97 - `reqwest` is now a plain dependency (it was Servo-only) — `blocking`, the
 98   same build cce-map and cce-calendar use; JSON bodies are serialized by hand
 99   rather than turning on its `json` feature, to keep it the same build.
100 
101 **The token** lives in the keyring as an entry with no `UserName`, found by
102 `service=raindrop.io`:
103 
104 ```sh
105 secret-tool store --label='Raindrop.io token' service raindrop.io
106 ```
107 
108 cce-keyring-sync skips entries without `UserName`, so it stays on this machine
109 and never goes to 1Password; the account index skips it too, so it is never
110 offered to a login form. A locked keyring is an error, never a prompt.
111 
112 **`cce-browser --raindrop-plan`** fetches Unsorted, plans a pass against
113 `bookmarks.tsv` and the base, prints it, and changes nothing. It runs ahead of
114 the single-instance hand-off, so it works while the browser is open.
115 
116 ## Phase 3 — the worker (done)
117 
118 `browser.raindrop` (KDL, `browser { raindrop (bool)true }`, off by default) is
119 read at launch and on every focus like the other settings; the worker thread
120 `cce-raindrop` starts the first time it is on and idles while it is off. Its
121 writing side is the "Sync Bookmarks with Raindrop" toggle on
122 cce-system-interface's Browser page (`src/pages/browser.rs` there) — keep the
123 key name in step with `settings.rs`. Choices:
124 
125 - **It polls the bookmarks every 2 s instead of being told about edits.** A
126   bookmark changes from the star, Ctrl+D, the bookmarks menu and the
127   `cce://bookmarks` page; comparing the in-memory rows (no I/O) catches all of
128   them with no hook in each. A change syncs once it has held still for one
129   poll, so a burst of edits is one pass. A full pass every 10 minutes brings
130   Raindrop's side; the page's "sync now" asks for one at once.
131 - **A pass's local half is one locked edit** (`Bookmarks::edit_rows`), so no
132   star or remove can land in the middle of it, and it is skipped entirely when
133   the plan changes nothing here — an idle pass never rewrites the file. The
134   worker re-reads the rows after its own pass, so its edits are not mistaken
135   for the person's.
136 - **Titles are flattened where they arrive** (`api::parse_item`): a tab or a
137   line break in a Raindrop title would otherwise be flattened by the TSV store
138   and read as "renamed in Raindrop" on every pass.
139 - **A skipped link edit leaves the base** (`settle_base`): otherwise its pair
140   would point at a URL that is not here and the next pass would trash
141   Raindrop's copy.
142 - **The status line lives on `cce://bookmarks`**, set by the worker through
143   `Bookmarks::set_sync_note`: what the last pass did and when, with "sync now".
144   A refused pass shows why and a **"sync anyway"** link carrying a random code
145   that `cce://bookmarks/sync-force` checks — a web page linking to `cce://`
146   cannot force a mass deletion, since it cannot read the page to learn the
147   code. The code holds for as long as passes keep being refused (a periodic
148   pass, a reload); a fresh one each time made the link on screen stale, which
149   is how the end-to-end run found it. After a forced pass the code is gone, so
150   reloading its URL forces nothing.
151 - The page is static: the line is as of the page's load, and reloading
152   `cce://bookmarks/sync` asks for another (harmless) pass.
153 - **Every change is logged by name** in `raindrop-sync.log` beside the base:
154   `time \t what \t link \t title`, one line per bookmark added, removed or
155   renamed on either side, plus refusals, forced passes and failures. Local
156   lines are the difference the edit actually made and Raindrop lines the
157   calls that succeeded, so the log says what happened, not what was planned.
158   Passes that change nothing write nothing; past 512 KB it keeps its newer
159   half. Added after the first day's fetch bug, when "which two bookmarks came
160   back?" had no answer — the status line only counts, and the process log was
161   gone with the restart.
162 
163 **Testing without an account:** `CCE_RAINDROP_API=<base url>` points the
164 client at a stand-in (it logs a warning when it does). The end-to-end run used
165 a ~60-line Python stand-in for Unsorted (GET/POST/PUT/DELETE, held in memory),
166 a throwaway keyring holding a fake token (the autofill section of CLAUDE.md has
167 the recipe and its two traps), and checked: first pass imports and creates; a
168 local remove trashes; a rename and a save on the "phone" arrive; an emptied
169 collection is refused with the bookmarks untouched; "sync anyway" runs it.
170 
171 ## Phase 4 — favorites (optional)
172 
173 A dedicated collection, ordered by Raindrop's manual sort.