It seems like the biggest downside of this world is iteration speed.
If the AT instagram wants to add a new feature (i.e posts now support video!) then can they easily update their "file format"? How do they update it in a way that is compatible with every other company who depends on the same format, without the underlying record becoming a mess?
Adding new features is usually not a problem because you can always add optional fields and extend open unions. So, you just change `media: Link | Picture | unknown` to `media: Link | Picture | Video | unknown`.
You can't remove things true, so records do get some deprecated fields.
Re: updating safely, the rule is that you can't change which records it would consider valid after it gets used in the wild. So you can't change whether some field is optional or required, you can only add new optional fields. The https://github.com/bluesky-social/goat tool has a linting command that instantly checks whether your changes pass the rules. In general it would be nice if lexicon tooling matures a bit, but I think with time it should get really good because there's explicit information the tooling can use.
If you have to make a breaking change, you can make a new Lexicon. It doesn't have to cause tech debt because you can make all your code deal with a new version, and convert it during ingestion.
That's true if you define the problem as "does my parser crash" and not whether the app is perceived as working correctly. If some platform adds support for video posts, then the next thing that happens is people start making posts that are only video. Meaning that in every other client, users see what appears to be an entirely empty post. Which will be considered a bug.
This is the core argument of Moxie's seminal essay, The Ecosystem Is Moving:
One of the controversial things we did with Signal early on was to build it as an unfederated service. Nothing about any of the protocols we’ve developed requires centralization; it’s entirely possible to build a federated Signal Protocol-based messenger, but I no longer believe that it is possible to build a competitive federated messenger at all.
That was written in 2016 but it was true then and continues to be true today. Users reject federated open platforms because the coordination costs mean they don't move as fast as proprietary centralized platforms, and they often appear broken even if technically working as designed.
Nothing about that analysis is unique to social media. It is also true of file formats. OpenOffice never took off because new features got added to Office first, so files that used those features would open in semi-corrupted ways in OpenOffice. The fact that OO represented things internally using open unions didn't matter at all.
I disagree that Bluesky is in conflict with The Ecosystem Is Moving. In contrast to most decentralized/distributed protocol projects they've managed to maintain control of almost all of their infrastructure with the exception of the personal data servers (pdses) of which they control 99.01%[1]
Almost all ATProto apps just fetch posts by handle => did:plc => post-type aka "lexicon", so they depend on what Bluesky decides to give them. If someone were to introduce unknowns into the flagship product's "lexicon" they could fix that at the API or Indexing level before shipping this data to the apps that depend on their API.
An actually decentralized network would have to overcome Moxie's criticism of the ecosystem. Can it be done? We'll keep trying.
Well, this doesn't prevent the "flagship" app from shipping things and doesn't slow it down. So it's at least not slowing down development which is the argument the parent post was making.
I've actually observed the exact opposite thing. Since Bluesky is open source, it's often visible when developers start working on a feature. And they often check in lexicon changes early on. As a result, there's been a few cases where third party client actually added support for these features earlier than the official one since they already knew the shape of the data.
This wouldn't always work, of course. Yes, if you're developing an app or a client, you better keep up with the ecosystem. But the landscape is competitive and there is no cost to switching. So if something falls behind, you can use something else.
Moxie's argument is that even if something has a flagship app, this doesn't help because if you use a new feature and then your friend complains that they can't see what you posted, the experience is just that the flagship app itself is broken. People don't experience this as, oh well, my friend should just have picked a better client. They experience it as, that's annoying, the video feature doesn't work reliably on Y but it always does on X.
An extreme example of this is WhatsApp and emojis. WhatsApp doesn't use the operating system's text rendering APIs to draw emojis, instead Meta license the Apple emoji font and draw the characters themselves. That's because if you do emoji the standards based, open way you get these problems:
• People use visual puns that only make sense if emojis look a certain way, without realizing they might look very different to other people.
• People use new emoji, without realizing that old operating systems can't draw them.
The experience in both cases is that it's simply broken and buggy. Version skew can kill platforms, which is why the most successful platforms today restrict which clients can connect and forcibly expire them on a regular basis.
BTW I don't think it's worth generalizing from Bluesky. Bluesky is an X clone whose unique competitive advantage is censoring conservatives even more aggressively than Twitter itself once did. It has no technical edge so they can develop the site as open source without it being a concern to them - they don't care if they leak what they're doing because technical innovation isn't a part of their brand to begin with - and the AT protocol clearly doesn't matter to more than a tiny fraction of its users. The point you're making in the essay is general, but ends up feeling overfit to Bluesky.
There is always one party "in control" of the lexicon and its canonical version.
I think it's important to distinguish this from the "every client adds their own features" thing. Technically yes, each app can add their own things to the open union that they support better. But it's also on each implementer's to consider how this would affect UX in other clients (e.g. if you add your own embed type, it seems reasonable to also prepopulate a link embed that acts as fallback). The problems you're describing are real, but I think we should give a bit more credit to the app builders since they're also aware that this is a part of their user experience design.
But still, whoever "owns" the lexicon says what's canonical. Then yes, some other software might not catch up to what's canonical but that's similar to what's happening with any platform that supports multiple clients today. Unless your outlook is that alternative clients in general are not competitive for this reason. I think that's a grim outlook, and if that were true, services wouldn't go to extra lengths to intentionally shut down their APIs, which so has been the trend with every network.
I think in longer term the bet is that the benefits unlocked by interop and a more competitive product landscape will become clearer to end users, who will be less interested in joining closed platforms and will develop some intuitions around that. This would not happen soon, so until then, the bet is that interop will allow creating better products. And if that doesn't happen, yes, it's pretty hard for open to compete.
Well, I never personally formed a strong opinion on Moxie's take, although I do understand it. Basically yes his outlook is that any service that doesn't actively ban alternative clients will be outcompeted by those that do.
The reason is that if alt clients are possible then some fraction of the userbase will adopt them. And if some users adopt them that means the experience of other users of the service gets worse, because new features become unreliable and flaky. You think you understand what another person sees and can do, but you don't, and this leads to poor experiences.
Viewed another way the client is an integrated part of the platform and not something that you can enable users to change freely, any more than they could be allowed to change the servers freely. We don't allow the latter because users would do things that broke the service, and so it is also for the former.
Empirically Moxie seems to be correct. Of the 90s era open protocols the only ones that survive are the web and email. The web survives but it's never been truly open in an extension sense - the definition of HTML has always been whatever the dominant browser of the era accepts, and this has become moreso over the time not less. There are no more plugin APIs for instance. SMTP survives, barely, because it's the foundation of internet identity. But many people even in corporate contexts now never send an email. It's all migrated to Slack or Teams. And if you look carefully at the Slack API it's not possible to make a truly complete alternative client with it, the API is only intended for writing bots.
This is grim but I'm not sure it's false and I'm not sure it can be changed. Also, Moxie's essay ends on a positive note. He observes that competition between mobile social networks does still work well despite the lack of federation, because they coalesced around using the user's phone number as identity and address book as the friends list, so you can in fact port your social network to a different network just by signing up. The notification center in the OS provides the final piece of the puzzle, acting as a unified inbox that abstracts the underlying social network.
This is rather mobile specific but seems basically correct to me. So that suggests the key pillar isn't file formats or protocols but ownable identity. It works because telcos do the hard work of issuing portable identities and helping people keep them, and ownership can be swiftly verified over the internet.
Maybe I’m optimistic for no good reason. I feel there’s something slightly different at play here but I struggle to put my finger on it.
Overall I agree with the take that a client for somebody else’s product is usually degraded experience. You want to deploy features vertically, and even existence of worse clients is a threat to that. I think one exception to that is when the main client is intentionally worse for the sake of other goals. Eg lots of ads. Or just the company being (or becoming) incompetent at making a client. Then alternative clients inject some new life and competition into the product. But if the product owner is good at making clients, they probably don’t benefit from fragmentation.
And still it feels somehow different to me. I think a part of it is that the space itself is vastly expanded in AT. We’re not just talking about a vertical app with a few offshoot clients that do the same thing but poorly. I mean, that still exists, but the proposition is that things can reach way beyond the original product. Products can blend. When products blend, there’s always this inherent coordination problem. Those who make multiple vertical products have to deal with similar things. At some point you have to stagger rollouts, support features partially or in a limited way, or show a simple version and direct to main app for more. Think Threads “bleeding through” Instagram, different kinds of cross-posting that they experiment with between apps. This is valuable but normally only monopolies can build meaningful cross-product bleeding because otherwise you have to negotiate API agreements and it’s just too risky to build on someone else’s turf.
What AT changes is that every product can cross-bleed into every other product. My teal.fm plays show up on my Blento page. So yes, you do have this fragmentation, but you also get something in return. It’s not fragmentation between the main way to show a thing and a couple offshoots, but between a web of products. And there is always a canonical schema so it doesn’t descend into chaos. I think there’s a “there” there.
I actually agree the key pillar is identity. It’s what AT URIs start with identity first. I think one way to look at AT is that the “contact list” analogy is right, but now that we’ve chosen the identity layer, we might as well put all our data into the contact list.
Riffing off the identity thing, one service I wanted for a while is something that issues X.509 certificates based on verified phone numbers. Phone numbers are a pretty great identity, perhaps the most successful private sector identity system ever, but they're expensive and annoying to verify, and the verification isn't portable across systems. A CA that did SMS verification and then gave you a certificate you could use with S/MIME or bind to passkeys or just use to sign software/documents in general, would democratize stable cryptographic identity. People generally can't handle key management directly, it's too easy to lose keys, but issuing transient keys tied to a phone number is much more palatable.
And PNs have got good features you want in general:
• Can have >1 of them if you want.
• Anonymous if you want.
• Not tied to any specific provider due to number portability laws.
• Hard to lose; phone companies will accept govt issued ID to get your account back if you lose your SIM and it's tied to a contract.
• Verifiable over the internet.
The only problem with them is they don't yield asymmetric keypairs.
The difficulty is business model. The people who want to consume such certificates are people who don't want to pay to verify numbers directly, but users don't want to pay either. So who pays.
Most apps reading records will validate a record against the schema for that type. e.g. there's nothing stopping you from making a app.bsky.feed.post record with more than 300 graphemes in the "text" field, but that post won't appear in the "official" app/website because it fails schema validation.
Similarly, there's nothing stopping you from adding another field in your post. It'll just get ignored because the app you're using doesn't know about it. e.g. posts bridged from mastodon by bridgy have an extra field containing the full original post's text, which you can display in your app if desired. reddwarf.app does this with these posts.
Lexicon validation works the same way. The com.tumblr in com.tumblr.post signals who designed the lexicon, but the records themselves could have been created by any app at all. This is why apps always treat records as untrusted input, similar to POST request bodies. When you generate type definitions from a lexicon, you also get a function that will do the validation for you. If some record passes the check, great—you get a typed object. If not, fine, ignore that record.
If the AT instagram wants to add a new feature (i.e posts now support video!) then can they easily update their "file format"? How do they update it in a way that is compatible with every other company who depends on the same format, without the underlying record becoming a mess?