mutuals index: tutorial part 2 #
part 2 is still todo!!!
part two will add an index of pre-computed mutuals for each account, eliminating the N point-reads for a single 100%-mutuals-hit-rate scan.
this gets tricky because conflicts become possible, since index entries for mutuals have to be able to be written by either party.
there are two ways we could add conflict avoidance, both which require a change to hubble-sync:
-
database-level conflict checking (optimistic or pessimistic). fairly straight-forward to add to the database engine crates, but we need to add a
Retryvariant to the main consumer-app callback error type to make it good. (we could desync on conflict, which is a very heavy kind of retry, but bleh. no.) -
app-level: use mutexes/semaphores to serialize writes at the application level, possibly with sharding. this almost works without changes to hubble sync, the lock has to be guarded until the commit returns, and you can't do that in the callback. we'd need to add a way to pass the owned guard back to hubble-sync, who could explicitly hold it until post-commit. useful enough to add, but this is a real step in diy-ing your own concurrency problems, meaningfully more than OCC from the storage engine itself.
going to add both of these to hubble-sync probably and they're not huge lifts.