The Lexicon Dilemma

I’m concerned that the atproto ecosystem is missing ways to foster new innovative lexicons, without sacrificing the network effect.

AT Protocol offers two great advantages for bootstrapping new apps:

• A new app can reuse the existing content to accelerate the usual flywheel starting problem that social networks face (the difficulty of getting users without content and vice versa)
• A new app can create any new lexicon to enable unique features while keeping the underlying platform, identity, follow graph, etc

These are both great advantages, but at the moment new apps can only really choose one, when they should be able to get both.

If a new app has a targeted niche or a low overlap with the existing content in the ecosystem, then it won’t be able to take advantage of that flywheel acceleration anyway. So for that subset of apps, it makes sense to only consider the lexicon that fits its use case best. For example we’ve seen some great lexicons come out around longer blog posts. Apps using these new lexicons aren’t likely to ever compete at the same scale as existing microblogging apps, but they’re not intending to. This group of apps are fine with this lexicon status-quo.

But there’s a much larger group of apps that would benefit from using the existing content. Even if they will only ever use a small subset of that content (e.g, an app that only shows movie reviews), being able to make use of what’s there allows them to skip some of the early hurdles a traditional non atproto app would face. If they want this they can just use the existing lexicons and jump straight in. With that decision though, that app is locked to this existing lexicon. They don’t get the same freedom of innovation. There may be new features which are ideal for the app, but are now harder to add (e.g, every post has a field referencing the IMDb ID, and an optional star rating).

Technically a new app could support both the existing lexicon and a new lexicon. The existing content it uses to kick-start the app won’t have the added fields or features of the new lexicon, but that is probably okay for the network benefit it will bring. But the whole idea of this new lexicon is to get some (or more likely most) content eventually created in it. This is what lets the app do some of the new interesting things it created this lexicon for. But just as the app has identified an overlap between its content niche and the existing corpus of content, there is also an overlap of users who want to view this type of content. Content creators will quickly see that, and now they need to choose whether to post in the new app and get the nice features but only reach one side of the viewers, or could post in the existing apps and reach both sides. Unless these features are so essential to the experience (in which case the app can’t take as much advantage of the existing content) this makes it a lot harder to convince creators to post in the new lexicon. The app’s best case may be that the user posts in both, but this gets messy quickly with attempting to deduplicate, unify likes, etc. And that’s before even taking into consideration the difficulty in convincing creators to cross post when they don't have to.

So if an app doesn’t want to go all in on their new lexicon and supporting both doesn’t solve their problem, then they could try to extend an existing lexicon. For example this movie app could put these new fields as custom embeds in the existing app.bsky.feed.post lexicon. This may seem like an obvious solution that I took a long winded path to get to. Many apps have already taken this approach and it’s worked great for them. For each of those, I see not only that it worked, but that it was the best option available to them, with no immediate downside. Unfortunately though, the best choices for the individual don’t equate to the best choices for the ecosystem; see the tragedy of the commons. If this approach is taken again and again, then we end up with several problems, but I’ll just highlight a few.

The first problem, and I’ll admit, this may just be my inner grouchy old man showing, is that the ecosystem shouldn’t just be locked into whichever lexicon was first, but should let them compete on merit. The app.bsky.feed.post lexicon is great at allowing generic posting that covers most post types found on other platforms, but like with any lexicon, it comes at this from a particular angle. For example, it assumes that the app should get to decide how the post is displayed. What if a format that allows the creator to say the image should go before the text, or the text should be overlaid onto the image, would work better long term? I’m not saying it would be, it's just an idea that an open social web that prides itself on giving creators more control could provide. The current situation makes it extremely hard for this new lexicon to get even close to a fair chance at being tested against the current lexicon. This effectively cements the Bluesky lexicon to be the atproto default forever. I don’t think this was the intended outcome by the folks at Bluesky, so I do hope they’ll be on board to help mitigate this.

Then we end up with problems around overloading and type detection. If the argument is that lexicon semantic drift is ok, then sure the movie app can reuse the Bluesky lexicon, but also other apps can use the embeds the movie app has defined. Is there a line to what extent this is or isn’t allowed? It would appear to be something we’d never get a clear consensus driven definition on.

• Can another movie review app reuse those?
• What about an app that does movie and tv reviews?
• Can an app to rate cat videos reuse the star rating embed only?
• Can a booktok app use the movie id embed to indicate movie adaptations of the book?

I could justify drawing the line to limit some of these use cases in a few different ways, but those wouldn’t really be consistent with the idea that the movie app should be reusing the Bluesky lexicon in the first place. It also doesn’t take much reuse for those to become complex quickly. The movie app also needs to identify which posts within this large app.bsky.feed.post corpus belong to it. Ideally it would want to show posts from other apps where similar reviews may be created. But once reuse of these embeds begins, distinguishing which posts have “correct” uses of these fields becomes more difficult.

The third problem I see is with context. Posting in this new app may add important context to the post that is lost when it's displayed elsewhere. Taking the movie app example, someone might post “I loved it” along with their rating and movie, which would make no sense when it shows just the text in the Bluesky app. What if it showed as a completely empty post, because someone just rated the movie and left no text comment? What if it looked even more controversial, with a post on Bluesky showing “They should have stuck to fossil fuels” without anyone realising it was posted about the movie Monsters Inc? The more overloading is done, the more this will occur.

These aren’t necessarily insurmountable problems, but each layer just seems to make this messier and messier. We’re in a great situation, still relatively early on in the hopeful great success of atproto. Changes made now are going to be easier than once this takes off even more. I believe we can solve this in a better way without resorting to just trying to patch holes this early on. I have been working on a proposal to address these issues, which I’ll share in a follow up post soon.