Setting up my own ATProtocol PDS

I’ve been interested in hosting my own PDS for a while ↗. The idea of being able to own the source of truth for my posts, uploads and social content is really appealing to me, and getting faster response times from hosting within the EU instead of the US-only Bluesky PDS servers is a really nice bonus.

The reason that I hadn’t made the move until now was for one reason: I really didn’t want to play sysadmin with my social presence. The idea of messing up and potentially losing my account/social presence was simply too offputting to want to risk it, especially as self-hosting in the ATProto ecosystem is still somewhat of a rough experience.

At the time I was making this decision, you could not easily migrate back to a Bluesky-official PDS. Once you made the jump to another PDS, there was no turning back. All of this meant that if I was going to take the plunge, I had to have a seriously good reason to do so. Luckily, a collection of lawmakers and payment processor executives across the world were about to give me just that reason.

The UK’s Online Safety Act ↗ (Wikipedia, sorry UK readers ↗) passed back in the prehistoric year of 2023. This law, in short, gives the UK government control over what is allowed to be seen online, and requires many online websites & social media to verify the age of their users to prevent “showing adult content to the wrong individuals”.

Now, I’m not going to go on about the implications of this law (that’s been covered well enough online) but it was definitely the push I needed to migrate away from the Bluesky-hosted PDS I was currently residing on. Staying would require me to complete the Bluesky age assurance process ↗, which would involve providing either a face scan or my ID to Epic Games, and I don’t want to do that because I care about my privacy and the free internet. Also, face scan models frequently assume I’m extremely young (thanks, I guess </3).

So, aided by my hate for the ever-decaying internet, I packed my bags and made for the land of self-hosted independence where I wouldn’t need to provide sensitive data ↗ to random companies (at least for now).

The first migration & the ATProto airport flight incident

Because I wasn’t knowledgeable or fully confident in my own self-hosting ability when it came to my online identity, or physically well enough at the time, I reached out to my evil demon friend Luna ↗, who is a very respectable sysadmin and had already been hosting its own account on a self-hosted PDS for some time.

After making some test accounts and ensuring account creation would succeed, I decided to use ATProto Airport ↗ to migrate my secondary Bluesky account over to Luna’s PDS. The process was actually really smooth and finished quickly, and there I was: self-hosted entirely away from Bluesky’s official PDS - bye bye, intrusive age verification! After this process, I also made my own rotation key for the account in case anything went wrong in the future (not foreshadowing, I swear).

Seeing the first migration go so smoothly, I decided hey, why not migrate my main account ↗, the process seemed really simple! Except… this time round, it was anything but.

Unfortunately, due to a bug in ATProto Airport ↗, the migration of my main account was a total disaster that took several hours and knowledge from some ATProto devs to sort out. I’ll be brief on the details here, as Luna has an amazing write-up ↗, including a fix should you ever end up in such a state (… please can we merge this ↗ already).

In short, when using the ATProto Airport transfer tool, the cookie which indicated the migration “target” was not cleared properly on logout. This led to the repository of the account I wanted to transfer (my main) being merged with the account I just transferred before (my alt), which meant that now a bunch of signatures failed to validate as the records weren’t all “mine”. Luckily, we managed to grab a CAR ↗ backup (basically a file of your account’s PDS data) and reconstructed the account again.

Once everything was back in “working order”, my current migration status was that I had one account on a Bluesky PDS (my main) and one account on Luna’s PDS (my alt). At this point, I took a few days away from trying to migrate my main account as I had to recover from the stress the whole incident caused.

A few days pass

After taking a break, I returned to the migration. I opted not to use ATProto Airport this time around for my peace of mind, but I wanted to mention that the developer has provided a very kind response ↗ and resolved the issue soon after (thanks!). Instead, I decided to use goat ↗, Bluesky’s own CLI tool that has account migration functionality built in.

Migrating via command line was really simple and felt way safer, my process was as follows:

1. Create a CAR file backup of my account from the Bluesky UI
2. Create my own rotation key via goat account plc add-rotation-key (own your identity)
3. Backup my blobs with goat blob export (not included in the CAR)
4. Backup my private account preferences with goat bsky prefs export (also not included, things like muted accounts)
5. Run the automated migration process with goat account migrate which does everything for you. If this failed at any step, I’d still be able to resolve issues manually from my backups.

All finished on the first try with zero fuss, with my digital social consciousness now being stored entirely on Luna’s PDS. I then decided to chill for a while and learn more about the inner workings of the protocol so I’d be more prepared to be my own sysadmin later.

Setting up a PDS of my own

⚠️ Do not follow this as your only guide for setting up a PDS, please refer to the official documentation ↗ instead, as change is rapid and I’ve made tweaks to fit my infrastructure.

After doing the digital equivalent of sleeping on Luna’s sofa for a few weeks, I was finally ready to make the move to hosting my own PDS. Thankfully for me, the entire self-hosting process revolves around Docker which I’m already familiar with and have my infrastructure setup with, for better or for worse.

When setting up, I mostly referred to the Bluesky PDS documentation ↗ for instructions. I didn’t end up using the provided PDS setup/install script as I already have my own setup and wanted to integrate the PDS with that, but it was still a really simple process (config env vars could do with better documentation, though).

compose.yml snippet:

[object Object],[object Object],
  ,[object Object],[object Object], at,[object Object],pds
  ,[object Object],[object Object], ghcr.io/bluesky,[object Object],social/pds,[object Object],[object Object],
  ,[object Object],[object Object], unless,[object Object],stopped
  ,[object Object],[object Object],
    ,[object Object], at,[object Object],pds,[object Object],data,[object Object],/pds
  ,[object Object],[object Object], <VARS ,[object Object], refer to documentation for up to date values,[object Object],
  ,[object Object],[object Object],
    ,[object Object], at,[object Object],pds,[object Object],net

Caddyfile snippet:

https://*.pds.dollware.net, https://pds.dollware.net {
    # Age assurance
    handle /xrpc/app.bsky.unspecced.getAgeAssuranceState {
        header content-type "application/json"
        header access-control-allow-headers "authorization,dpop,atproto-accept-labelers,atproto-proxy"
        header access-control-allow-origin "*"
        respond `{"lastInitiatedAt":"2025-07-14T14:22:44.912Z","status":"assured"}` 200
    }
    reverse_proxy <UPSTREAM>
}

When creating my DNS records, I also decided to temporarily rent a floating IPv4 address for my PDS server as Cloudflare’s free tier does not allow you to make certificates for subdomain wildcards (e.g. *.pds.blooym.dev). I didn’t want to expose my server’s regular IP address by disabling proxying during setup; I can always dispose of the floating IP later and enable proxying for the PDS itself once all my accounts are set up, as I already use a custom domain handle (you probably shouldn’t do this btw).

Once I had everything started up and configured properly, my PDS was available to the world! Now all I had to do was make a test account to ensure things worked and, if that went well, migrate my accounts all over again.

I created a test account via the Bluesky UI with an invite code I generated via pdsadmin.sh create-invite-code (which by the way, is a really weird admin script, it makes a web request + requires root ↗ for each command). It showed up properly and all interactions seemed to be working as intended, so I requested bsky.network to crawl my PDS with pdsadmin.sh request-crawl and decided to once again do my migrations - starting with my alt account, which, at this point, is somewhat of my test case for these things.

I reran the exact goat CLI steps I mentioned earlier for my main account, and the migration was eventually a success. I did have some weird issues with importing my data a few times around, but those eventually went away by themselves, so I’m not really sure what happened.

At long last, I had my alt account on my own PDS and everything was working properly, so I said my goodbyes to Luna’s online sofa and migrated my main away. It’s at this point I thought I was finally free to call it a day and go back to looking at yuri and talking to my mutuals, knowing that I’d finally finished The Great Migration.

The start of a long headache

About 30 minutes after finishing up my migrations and using Bluesky as usual, I noticed that my posts had stopped making their way to the wider Bluesky network via the relay. I was confused, I hadn’t changed anything and yet… everything had suddenly fallen apart?

I immediately complained to some friends online and then jumped to debugging. I tried:

• Making sure my PDS is accessible and that all DNS records are resolving and SSL is working - check.

• Testing record creation on my PDS was working using PDSls ↗ to query my PDS directly - again, check.

• Ensuring all WebSockets were working properly using the goat CLI once more - my PDS was properly emitting new events when I used goat firehose wss://bsky-pds.blooym.dev/xrpc/com.atproto.sync.subscribeRepos and making new records - so again, check.

• Running a restart of all the servers involved in hosting my PDS - no change.

At this point, I’m completely lost: nothing is wrong with my PDS configuration or setup, everything had been working fine beforehand, and now the Bluesky network just didn’t know I existed. Whenever I made any post and tried to look at it without querying the PDS directly, I’d be hit with

which led me to believe that the problem lied elsewhere. I started trying to diagnose if the problem was on Bluesky’s side:

• I made a call to XRPC method com.atproto.sync.listHosts on all Bluesky firehose services to see if my PDS was listed, it was not (I’m still not sure if this means anything).

• I used goat firehose once again, but this time I used one of Bluesky’s Jetstream ↗ instances as these show everything that happens on the network. Once connected, I made a record on my PDS via the Bluesky UI and noticed that it… simply did not make its way to Jetstream.

At this point, I knew for a fact the problem was between my PDS and Bluesky’s wider network, but I was at a total loss for the why or what. At this point my setting up and debugging my PDS had taken over 3-4 hours of my life away and it was getting really late (well, 11pm), so I resigned to posting a question in a place where some ATProto regulars would see it and went to bed.

The next morning

I woke up bright and early at 6:30am, and, before I even had a chance to open my eyes properly, I jumped straight to testing once more. I reran all my debugging from last night and… nothing had changed - I was still completely invisible to the network.

At this point, I’m kind of pissed off, which I think is a totally valid reaction to all of this. I was debating writing a ticket to Bluesky to ask if it was something on their side as I’d read in some issue trackers & channels online that this had happened to others and I was not the first. But before letting my hopes die off and sending that ticket, I wanted to debug one. last. thing: DNS.

I’ve been using Tailscale ↗ for a few years now and have really liked it - it has made networking across my home and cloud servers super simple and being able to just hop in to some private things I host at home from anywhere in the world without messing with WireGuard is great.

Tailscale uses something called “Magic DNS” ↗ to make some of the machine <-> machine networking easy - instead of remembering a Tailscale machine IP, you can simply use the machine name. For example, instead of using 100.70.225.10 (which could change later), you can simply use sartha.mech-pilot.ts.net (or hell, even sartha alone) to resolve to a Tailscale-internal IP instead.

Cool, sounds great, and I’ve been using Magic DNS for a long time without issue. With all that said, if I know anything about magic, it CANNOT just be “magic” forever, and all great tricks must eventually fall apart.

Remember that little PDS problem? The one where the events that I made didn’t make it to the Bluesky network? You know, the problem I lost hours to? The cause of all my problems was… you guessed it… Tailscale’s “Magic” DNS. How did I figure this out? I wish I could say something deep, but instead I will simply tap the sign that says “It’s always DNS” and move on.

So, my hours spent figuring it all out came down to just running tailscale up --accept-dns=false on the PDS host server, sudo reboot to clear up all the state, pdsadmin request-crawl one last time to get Bluesky to acknowledge me, and…

Everything was working well once again, and all the events between when the problem started and now were sent out as they should have been originally. Thanks, sequencing.

Or so I thought, until 2 days pass

Suddenly, while casually using Bluesky, my interactions stop making it to the network again. At this point, I had already published this post detailing my process, and I’d pointed the finger at Tailscale mDNS as being the cause of my problem. I was really confused - I hadn’t dared to touch any part of my infrastructure as I was still burnt out - but once again, I was having the exact same problem.

I immediately had one thought: “what if it was Cloudflare’s proxying?”. During my previous troubles, I hadn’t enabled Cloudflare’s proxying feature on my PDS’s domain, so I wasn’t sure that turning it off would achieve anything - after all, if it was broken before with it turned off, why would it being on cause problems now?

Regardless, I allocated another floating IP to my server, disabled proxying on my PDS’s domain once more, and restarted my server. I know Cloudflare has had issues with WebSocket handling in the past, so perhaps this was actually a multi-layered problem?

Immediately after rebooting my server, I requested another crawl from the Bluesky network, and I saw the User-Agent “indigo-automod” in my logs very shortly after, which indicated to me that my server was visible to the network once more and that the problem had once more been resolved. At this point, I am more than lost with regards to what’s going on and what the cause is; I don’t want to point the finger at anyone or anything, it feels like my PDS is barely holding itself together when it comes to network communication.

“So, it turns out the problem was actually Cloudflare proxying then?”

Ahaha! No. Because the day after fixing the problem for the second time, I woke up to find that, once again, I was experiencing the exact. same. problem. The two prior “fixes” I found turned out to have been misleading red herrings and not the problem at all.

At this point I’m starting to have flashbacks to the “insanity monologue” from Far Cry 3, the one where Vaas Montenegro says to the player “Insanity is doing the exact… same fucking thing… over and over again, expecting… shit to change.”

I tried everything I could to fix the problem again, but this time around absolutely nothing was working. Several days passed where the problem would not resolve, so I decided it was time to start investigating Bluesky’s hosted pieces of the ATProto network - I was entirely sure that this wasn’t actually my fault.

A broken relay

Before getting into this section, here’s what you need to know: A relay is an implementation detail that connects to thousands of PDSes in the network and forwards all the events it receives into one big firehose that applications can build on and use. There’s more to it than this, but that’s the basic gist. If you want to learn more, check out this page from the Bluesky team ↗.

Rather than chronicle every single step of my troubleshooting process, I’ll cut to the chase: I tested against Bluesky’s new relay implementation (which at the time of writing wasn’t being used by the app itself) and some community-hosted relays, primarily Microcosm ↗.

To speed up testing how events from my PDS fanned out to multiple relays at once, I created a CLI tool called Relayhound ↗ that automatically tails the output of multiple PDS/relay websockets simultaneously and marks which ones output the desired data.

My testing confirmed my suspicion: the issue was with bsky.network specifically, not the other relays (ignore atproto.africa, it was in a weird state during testing). While the other relays on bsky.network subdomains ran a newer codebase simply called “relay”, the apex domain bsky.network actually ran an older implementation called BGS. Since Bluesky’s AppView was powered by this older BGS relay at the time, it was dropping my events even though the other two relays worked fine.

With no way to fix this myself, I opened a support ticket on August 22nd. After some back-and-forth, I received this response on September 9th:

“We think we’ve identified and fixed the issue - the relay db was in an inconsistent state, which has been manually corrected for now (it has a small chance of happening every time a new PDS comes online). Could you try posting something again?”

- Bluesky Support

I tested it immediately, and it worked! However, I was still missing a few weeks of posts and likes since the app didn’t backfill the blank period. Fortunately, the developer of PDSls ↗ added a feature that let me manually recreate existing records, which re-emitted the events with the original timestamps.

After this, my problem was actually solved. No more debugging, I was finally free to just use the app normally again.

It’s actually been a while since this issue was resolved, and in the time since then I got some additional context from a former member of Bluesky’s engineering team: the “inconsistent state” mentioned was caused by a bug in the relay implementation that bsky.network runs on. When a new PDS is discovered and crawled, there’s a chance it can be inserted into the relay’s database twice, leading to an inconsistent state where the PDS is either skipped over or maintains two different states simultaneously. Definitely a strange bug, it is mildly concerning that this could be impacting other new PDSes.

Conclusion

This whole experience was exceedingly stressful. While I’ll still recommend others set up their own PDS or migrate off the Bluesky ones, I also want to warn you that by doing this you may randomly run into issues in parts of the network you don’t control, and you will, in part, take on the responsibility to resolve them.

The ATProtocol may be open, but if you want to interact on Bluesky, you will be at the mercy of their infrastructure and have to hope nothing breaks there, as you’ll depend on them to fix it. The benefit to being on a Bluesky-hosted PDS is that any issue that may arise is far more likely to be resolved quickly since it’ll impact a very high portion of their userbase, and you likely won’t have to inform them of the issue yourself.

Regardless of all this, I’m planning to continue using Bluesky and the wider ATProto network in some capacity for the foreseeable future. Most of my friends are on there, and I’m interested to see where things like Tangled ↗ and other ATProto applications go in the future.