A simple DAV sync application / cli

Files
README.md

davsync #

Two-way synchronisation of local folders with WebDAV collections. It runs as a background service that syncs each folder at its own interval. A setup window (Chromium, opened from the command line) is where folders are configured and where you can see what was synced and why.

davsync                    # the window: add and configure folders, sync on demand,
                           # inspect state and activity
davsync service            # the service: sync every enabled folder at its interval
davsync sync [folder...]   # sync once and exit
davsync plan [folder...]   # show what a sync would do, without doing it
davsync status
davsync install            # start the service whenever you log in
davsync uninstall          # stop doing that, and stop the service

Install #

Install bundle if you don't yet have it. Then click the link to install davsync: nzip:pipobscure.com/davsync

Or, from a terminal:

bundle install @pipobscure.com/davsync   # puts davsync in ~/.local/bin, kept current by `bundle update`

install uses what each system has for this:

  • Linux: a systemd user unit, ~/.config/systemd/user/davsync.service. It is enabled for default.target and restarted if it fails. Logs: journalctl --user -u davsync.
  • macOS: a LaunchAgent, ~/Library/LaunchAgents/com.pipobscure.davsync.plist. Logs: ~/Library/Logs/davsync/davsync.log.
  • Windows: a script, %LOCALAPPDATA%\davsync\davsync-startup.ps1, and a shortcut in the Startup folder that runs it hidden. A .ps1 placed in the Startup folder itself would only open in Notepad. Logs: %LOCALAPPDATA%\davsync\davsync.log.

Each starts this installation's own bundle with this node, by their absolute paths, so it does not depend on PATH. Install from where the bundle will stay (bundle install puts it in ~/.local/bin). install --print shows what it would write.

Needs Node 26.11.1 or later, and Chrome or Chromium for setup.

Passwords #

Passwords are kept in the system's own keychain, never in the config file:

  • Linux: the Secret Service (GNOME Keyring, KDE Wallet) via secret-tool, or KDE Wallet via kwallet-query.
  • macOS: the login keychain.
  • Windows: Credential Manager.

A config written by an earlier version, with passwords in it, is moved over the first time it is read. DAVSYNC_KEYCHAIN=file keeps them in a file only you can read instead, for machines without a keychain.

When it cannot sync right now #

Some things pass by themselves. When one of them stops a sync, that sync is skipped, recorded with the reason, and tried again at the next scheduled time. It does not fail, and the wait does not grow:

  • the keychain is still locked (a service started at login, before you unlocked it) or not reachable;
  • the network is gone: DNS, refused or reset connections, timeouts;
  • the server says it cannot answer right now (408, 429, 502, 503, 504).

If the connection drops in the middle of a sync, the rest of that sync is not tried; the next sync picks up where it stopped. Everything else, such as a password the server refuses or a local folder that has gone, is a failure, and repeated failures space out the service's attempts (up to eight intervals).

How a sync works #

A sync compares three states of every entry:

  • last-seen: how the entry looked on both sides when they last agreed;
  • local: what a scan of the folder finds now;
  • remote: what PROPFIND reports now.

Each state records the entry's location, size, mtime and ctime, the SHA-256 of its content and of its first 4 KiB, the etag, the local and remote URL, and an identity hint (the inode locally, oc:fileid on Nextcloud/ownCloud).

src/sync/ splits the work into steps that don't know about each other:

Step File Does
scan local.ts, remote.ts lists both sides; no content is read
detect detect.ts per side: unchanged, modified, added, deleted, or moved
plan plan.ts turns the two sides' changes into actions; no I/O
execute execute.ts carries the actions out, settling the last-seen record of each one that succeeds

engine.ts runs the steps in order. state.ts keeps the last-seen state, and a log of runs and actions, in SQLite.

Recognising changes, cheapest evidence first #

  • A local file whose size, mtime and ctime are as last seen is trusted without reading it. The exception is a "racy" one, modified less than 2 s before it was recorded (git's rule). If its size changed, it is modified. If only its timestamps changed, the 4 KiB head usually settles it. Only an equal size and an equal head make it read the whole file.

  • A remote file is judged by its etag. A changed etag with an unchanged server checksum (oc:checksums SHA256) is no change. Otherwise the content is unknown until it is downloaded, and the download hashes it as it arrives. An etag that changed over identical bytes is recorded, not written.

  • Moves are recognised, most certain first:

    1. by identity (inode / file id);
    2. because their directory moved;
    3. by content: SHA-256 locally; etag, checksum, or (for files of 4 KiB or less) the head remotely;
    4. a directory counts as moved when everything below it turned up below one new directory.

    A move becomes a MOVE or rename on the other side, never a delete plus an upload, and a directory moves as one operation.

Deciding what to do #

  1. Nothing is lost. A deletion never wins against a change on the other side. When both sides changed a file differently, the local version is kept as name (conflict <date> <time>).ext on both sides, next to the remote version.
  2. Both sides renamed the same thing differently: the local name wins. The content is the same, so no data is at stake.
  3. Two things never end up at one path. A move onto a path the other side has just started using is downgraded to a delete plus an add, and then handled as a conflict.

Being careful #

  • Every write is conditional. Remote writes use If-Match / If-None-Match. Local writes re-check the file's stat right before replacing it, and downloads land in .davsync/tmp and are renamed into place. Anything that changed after the scan is skipped and picked up by the next sync.
  • Files deleted locally because of a remote deletion go to .davsync/trash/ inside the folder, and are kept there for 30 days.
  • A side that comes back empty when entries were synced before (an unmounted disk, a wrong URL) is refused rather than mirrored.
  • A scan that cannot read part of a side aborts. A partial listing would look like deletions.
  • Only one sync per folder runs at a time, across processes.

Repository layout #

The root package is the app. packages/ holds the WebDAV client, the WebDAV server engine (used by the tests as a real server), and the XML parser they build on. All three are copied from ordner.fyi and are npm workspaces here. They are built to dist/ on npm install.

src/main.ts           CLI
src/service.ts        the periodic runner
src/config.ts         folder configuration (owner-only JSON; holds passwords)
src/sync/             the sync engine (see above)
src/ui/               the setup window: Chromium over CDP, as in bled
web/, public/         the setup page
packages/             @ordner.fyi/xml, observe, dav-core, dav-client
test/                 detection/planning unit tests; end-to-end tests against dav-core
tools/                pack, smoke test, release
.tangled/workflows/   CI and release, on tangled's spindle

Development #

npm install                 # also builds packages/
npm test                    # typecheck + tests
npm run test:packages       # the workspace packages' own suites
npm start                   # the window, from source

Packaging and releases #

The app ships as one file, davsync.nzip, built with @pipobscure/bundle. It holds the app's sources and every module it runs, behind a launcher, so it runs by name:

npm run pack      # build/davsync.nzip (members dated with the commit, so a rebuild
                  # holds identical members; identical archive bytes need the same node build)
npm run smoke     # run it by name: --version, and the setup window headless
./build/davsync.nzip

CI and releases run on tangled's spindle (.tangled/workflows). ci.yml tests, packs and smoke-tests every push and pull request to main.

To release, set version in package.json, commit, git tag -a v<version>, and git push --follow-tags. The tag starts release.yml, which runs tools/release.sh. That script tests, packs and smoke-tests, attaches davsync.nzip to the tag as a release file, vouches for it with an attestation (bundle attest --kind published), and publishes the davsync listing. Then:

bundle install @<account>/davsync

It needs the secrets TANGLED_HANDLE and TANGLED_APP_PASSWORD (tangled secret set …). Run by hand (tangled run trigger -w release.yml), it is a dry run. The release job gets the tangled CLI with bundle install, and accepts it only with its publisher's attestation (ci/policy.json).

License #

EUPL-1.2