Today (2024-02-07) I'm taking a look at the [published whitepaper](https://arxiv.org/pdf/2402.03239.pdf) for Bluesky's AT Protocol. I'll be referencing my own encoding format, [`fog-pack`](https://crates.io/crates/fog-pack), pretty often. Overall I'd say it's quite promising, but has avoided any of the real federated system trials so far by *not federating the aggregator*, and thus hasn't been truly put to the test yet. It's supposed to be an improvement over Mastodon's federation scheme, and is intended for higher decentralization. The actors in the system appear to be: - Users, who have clients that talk to a server - Servers that store client data - Relays, which aggregate server updates into a "firehose" - Labelers & Feed generators, which process the firehose into useful sub-information - Tallying up all likes and re-posts and such is done by an "app view" Mastodon merges the server storage, relaying, and processing into a single server. Bluesky, so far, has only pulled the processing part out to enable custom feeds. ### Data Storage Data storage is done with CBOR, specifically DAG-CBOR because the ATProto guys worked out the same thing I did years ago: you need to have a canonical encoding so that document hashing works. Great! They *also* have what they call lexicons, which is analogous to fog-pack schemas: strictly defined data structures for interacting with a system. Again, great! We're approaching fog-pack stylings here, which makes me feel that I've been on the right track with fog-pack. The Personal Data Server (PDS) is a merkle tree (good) and can reasonably be stored on any server, self-hosted or otherwise. The trick is convincing a relay to pull PDS records from your server - Bluesky does not yet enable this, which I find unsurprising because it's equivalent to federating and is thus one of the biggest decentralization pain points. ### Moderation Moderation is what still looms large here: PDS hosts will want to look at all your data, and may even need to in order to comply with local laws. That means processing all your data through a moderation algorithm. A labeler would be the moderation algorithm and would then need to loop back around to excise bad PDSes from both the relay server and whatever PDS host is running said labeler. ### Unique Identifiers ATProto uses [Decentralized Identifiers (DIDs)](https://www.w3.org/TR/did-core/), a relatively recent W3C standard that I recall not liking. Looking at it again, that'd because it mixes in the ideas of a document store, how documents can be retrieved, how documents can be updated, what methods are used to make the unique identifier, and who's in charge of each collection of DIDs. Probably because this is meant to encompass many different approaches to the same thing. *Except* that it has the annoying effect of "specifying" a standard without stating any specifics. What's the document representation? Could be many things! ATProto assumes JSON, so I guess you're hosed if that's not what someone else assumed. Some scattered notes on DIDs: - Your account is uniquely identified by a DID - the handle is just a shortcut for people to use, and a way for humans to associate you with a known organization. - ATProto assumes your DID is a JSON document - Your DID is either stored in a DNS TXT record, or returned from a "well-known" URL for a domain name. - Figuring out your Handle is done by claiming a specific handle in your DID document, and that handle resolving to your DID. - ATProto rightly calls out the DID standard for supporting "hundreds" of resolution methods, which of course sucks for an implementer like themselves. So they only support `web` and `plc`, with the latter being their own resolution method ATProto's DID storage and retrieval seems like a OK initial attempt to store a unique public key alongside a claimed handle. Updates, of course, are a bit finicky and aren't satisfyingly addressed here. I think the Fog way of doing this, with fully user-handled decentralized certificate stores, might work better. Or at least they could make these DIDs have an expiration time so you could detect a malicious server that never updates a DID document. Bluesky stores key material themselves, not trusting your average user with their own private key. ATProto itself doesn't care who has the private key though. ## Lexicons [Discussed here](https://atproto.com/specs/lexicon). Extremely similar to fog-pack. There's a number of types: - Same as fog-pack: `null`, `boolean`, `integer`, `string`, `bytes`, `array`, `object`. `cid-link` takes the place of fog-pack's `Hash`. - There's also `blob` and `param`. - `blob`: A byte sequence with an associated MIME type. - `param`: Only used with queries, and used to limit the number of available types inside an `object`. Think of it as an object with a predefined set of permitted value types - A number of "meta" types: `token`, `ref`, `union`, and `unknown`. `ref` is like a fog-pack `ref`, `union` like the fog-pack validator `any_of`, `unknown` is like the fog-pack validator `any` . `token` is just a string that's assumed to be special. It seems like a fog-pack `enum` where no attached data is allowed. - They also have `query`, `procedure`, `subscription`, and `record` as special types, which are really the enclosing protocol concepts. - They specifically do not have floating-point values - They do not break out public keys, encrypted data, or private keys (encrypted for transit or otherwise). The only cryptography here is the `cid-link`, which is a cryptographic hash, and CBOR's enclosing encryption capabilities - [COSE](https://cose-wg.github.io/cose-spec/), which supports object signing and encryption. - They do specify public/private keypairs, and public keys may be encoded as special DID strings. - Timestamps are not innate, but can be an additional special string type (see below) - Compression is likewise not directly handled in the spec Strings hide additional special types. The below is directly taken from the spec (with minor modifications): - `at-identifier`: either a [Handle](https://atproto.com/specs/handle) or a [DID](https://atproto.com/specs/did) - This is how public keys are propagated - `at-uri`: [AT-URI](https://atproto.com/specs/at-uri-scheme) - `cid`: CID in string format, details specified in [Data Model](https://atproto.com/specs/data-model) - `datetime`: timestamp - "full-precision" times, which must meet ISO 8601, RFC 3339, and WHATWG HTML datetime standards. They don't support subsecond times very well, and can experience roundtripping issues. - `did`: generic [DID Identifier](https://atproto.com/specs/did) - `handle`: [Handle Identifier](https://atproto.com/specs/handle) - `nsid`: [Namespaced Identifier](https://atproto.com/specs/nsid) - `uri`: generic URI - `language`: language code - Should comply with [BCP 47](https://www.rfc-editor.org/info/bcp47), defined in [RFC 5646](https://www.rfc-editor.org/rfc/rfc5646.txt). The "primary types" are records, procedures, subscriptions, and queries. There's a fair bit of similarity to fog-pack documents, entries, and queries. They *are* different though - procedures are akin to HTTP POST, which fog-pack specifically does not support, since it doesn't assume there's a distinction between servers and clients. POST only really works when you assume there's some notional server that you'd post things to. Likewise, fog-pack doesn't distinguish subscriptions and queries - all queries return an ongoing subscription, since records may change at any time or be different between many different nodes. If you always have a specific server in mind, like ATProto, you can distinguish the two. ### Concluding Thoughts Overall ATProto has a lot of the same ideas as fog-pack, so I'm feeling pretty good about fog-pack's prospects. Adapting pieces from the ATProto lexicons that Bluesky has made could be quite useful for making a social network. ATProto has a decent shot at going from the current centralized social network model towards a more federated one, though I think the Relay portion is going to be the toughest thing to federate, and will be the real trial for the protocol's future.