Tangled is considering to change the data host of few records from PDS to Knot; a headless git server designed to be PDS-like service for git repositories. I think this is one of the cases where current atproto PDS model doesn't fit well even with upcoming permissioned space.

This article has two purposes.

1. explain why Tangled is going to store this data in Knot rather than a PDS while atproto is moving torwards permissioned space model.
2. elaborate the need for a cross-service record referencing syntax in AT-URI since we want to reference a record-like data stored outside of PDS.

Contribution-driven Data #

From last post, I used term "collaborative objects" (COBs), but I think "contribution" explains the core problem more clearly, so I'll keep the acronym "COB" and read it as "contribution objects" from here. "Contribution" is different from "collaboration": it means there is a specific target you are contributing to. It's about "you to them", not "we".

It's a bucket! #

From permissioned data diary 3, Daniel mentions two options: colocated where the group's data is stored on alices PDS and everybody contributes to it there, or partitioned where the data is spread out across all contributor's PDS and Permissioned Space is about second option.

But, in case of Tangled and few other applications, first option fits better. In this case, it isn't "Your Bucket, My Data" but "Your Bucket, Your Data". Daniel renamed the term "bucket" to "space" from the following post because "space" fits better when data is partitioned across multiple hosts. But for contribution-driven data where data actually lives under one identity, it's not partitioned so "bucket" actually gives pretty good intuition over "space".

For Tangled, this bucket is a git repository and Knot is the service that holds it, so I'll use "bucket" and "git repository" interchangeably throughout the rest of this article.

When does bucket model make sense? #

The data deletion example explains the difference between space and bucket models quite nicely. When alice deletes all her posts from "protocol nerds" AtmoBoards forum, the forum is expected to delete all her posts. But when alice tries to delete her issue from "tangled.org/core" Tangled repository, well, alice shouldn't even be able to do that. Alice already contributed to that repository and her valuable work should not be undone. Alice can unlink her identity from the created issue, but the issue itself, and all related conversations are expected to remain. This is similar to "deleted" comment situation from Reddit.

While this might be an expected behavior for community forums like reddit, it doesn't really make sense from contribution-driven platforms like git forge or community wiki. You can't delete your contributions. Once you contribute, you are not the only one who own it. Merge state of contribution doesn't matter much because issues can be considered as a contribution by their existance. It represents that there is a bug or a task. Even comments are treated as contribution representing (hopefully) helpful discussion around issues and tickets.

So, when you contribute, both you and target bucket own the data, and bucket's ownership should live longer.

How does it look like? #

There are two identities in bucket model:

1. Contributor
2. Target repository (=bucket)

We currently assume target repository to be Tangled git repository, but imo it can be anything that accepts contributions like community wiki. When contributor writes a COB record to the target repository, they will directly write a record in Knot and attest it by creating a hard-reference record in their PDS.

Wait, isn't that public write access? are you sure?

Yes, I'm not crazy. And it's not that crazy of an idea if you think about it. When you self-host a traditional git forge like gitea or gerrit expecting contributions from anyone, you are literally opening write access to public network. If your forge and repository are truely public, anyone should be able to signup a new account and create issues. As a maintainer, you will want to hold those contributed data because that's literally what you are asking for: "Please contribute to this repository". If you don't want that, you can simply gate it by custom ACL or completely lock down the repository only for permissioned users.

Shouldn't you aggregate contributions then? #

If you are familiar with atproto, one idea you might come up with is to aggregate contribution records from all contributor's PDS, creating hard-copy into repository's bucket.

Short answer: we don't want to and it doesn't differ much.

Imagine you maintain a small open-source project expecting contribution from public network. As a maintainer, you will always want the hard copy of contributed data because you should be able to keep it even if original author's account get completely nuked. To aggregate contributions from distributed PDSs, you should ingest the entire firehose which is quite expensive operation. Also, even if you aggregate, that doesn't mean you are free from spams. Regardless of you aggregate or just open up direct writes, you will eventually need a way to gate and moderate them anyway.

What we need from upstream atproto spec #

Record-like data can live outside of the atproto_pds, so we need a way to reference it. We don't need COB support in atproto because we already solved that via Knot and we think having external source like this makes more sense because obviously PDS will never host git repositories. Instead, we need a reference spec we can use outside of the PDS.

COB is still a record. It has an owner (the bucket, identified by DID), a collection type (NSID), and a record key. It just lives in Knot rather than a PDS. So it seems like we can use AT-URI to reference it, except AT-URI has no way to point outside the PDS today.

PDS record AT-URI:
at://{authority}/{collection}/{record-key}

Knot record AT-URI:
at://{authority}/{collection}/{record-key}?service=tngl_knot
at://tngl_knot@{authority}/{collection}/{record-key}
at://{authority}/tngl_knot/{collection}/{record-key}

I wish IETF working group can figure out a better method to specify a service namespace in AT-URI.

This is the upstream change contribution-driven applications need from atproto: an AT-URI that can point at record-like data living outside the PDS.