Repo writes are currently serialized with an in-process per-user mutex
(repo_write_lock.rs),
acquired in begin_repo_write
(repo_ops.rs L182)
and held through the commit.
This works with one Tranquil process, but not with multiple instances sharing the same Postgres database: each process has its own mutex map and can therefore write to the same user's repo concurrently.
The database still protects the repo from inconsistent concurrent commits. apply_commit locks the repo row using FOR UPDATE NOWAIT and checks expected_root_cid
(postgres/repo.rs L1376-1404).
The problem is that this turns normal concurrent writes into client-visible failures. NOWAIT fails immediately and ConcurrentModification is returned as InvalidSwap
(repo_ops.rs L75-80),
so one of two ordinary writes can receive Repo has been modified even though it did not use swapCommit.
With a single process, the in-process mutex prevents this by queueing the writes. With multiple processes it cannot.
Background work also appears to assume a single process. start_scheduled_tasks, the startup backfills and the Signal client are started by each instance
(main.rs L196-204,
L317).
These likely need their own coordination before multiple instances can safely run against the same database.
Sequence assignment already uses pg_advisory_xact_lock
(postgres/repo.rs L1055-1066),
so for the Postgres repo backend a similar approach may make sense for repo writes.
I don't think this is necessarily the only thing required for multi-instance operation. Blob storage, cache behavior, background tasks and the different repo backends will need to be considered separately.
Until then, it would be useful to document that running multiple Tranquil processes against the same database is not currently supported, including during rolling upgrades.
