Possum - Querying public data from your PDS

Posts

I really like the idea of having a place on your personal website / digital garden where you can just share your thoughts and opinions. When im visiting someone's blog I always look for a /posts somewhere in the page. Please tell me more about yourself, and what interesting projects you have been working on!

I tried to make something similar during the first iteration of my website: During the build process of the my pages I also parsed some djot files into html. It worked alright for a while, but figuring out how to make syntax highlight work on my code blocks was not worth the effort at the time. Thats why I started using Leaflet for my blog posts instead.

standard.site

Leaflet feels really nice to use. The text editor UX is great and code blocks work no matter what language im decide to write. The only problem is that now I dont have direct access to my posts: they are not stored on my website's codebase anymore, they live in my Personal Data Server (PDS), together with the rest of my records.

The good news is that its all public. Maybe, if I could find a way to access them, I could display my list of posts again! They all use the standard.site.document lexicon definition, the fields are well documented and can be easily turned into a valid Gleam type. After some research I found that I can make HTTP requests directly to my PDS in order to query the data stored in its records and receive a response as JSON. After I learned how to use their API, all I needed was to parse the body using the decode module.

const url = "https://bsky.social/xrpc/com.atproto.repo.listRecords"
  <> "?repo=kacaii.dev"
  <> "&collection=site.standard.document"

fn fetch_documents() -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], document_decoder())
    use documents <- decode.field("records", decode.list(value_decoder))
    decode.success(documents)
  }

  let handler = rsvp.expect_json(decoder, ApiReturnedDocuments)
  rsvp.get(url, handler)
}

Good enough, now I can continue writing on Leaflet peacefully, and my website will take care of displaying a preview of my latests posts, including the one you are reading right now.

Possum

It's just a HTTP request targetting an endpoint, the concept being simple but takes a bit of setup in order to do it correctly: You need to know what's your Decentralized Identifier (DID) and you also need to know where your PDS is located. If I can take care of that, I believe this could help other projects. There are quite a lot of gleam developers interested in the AT Protocol but not many atproto related packages, I want to make the process of accessing that dataset easier for anyone that is interested in learning more about ATproto.

I decided to make a library for building this kind of requests. First I needed a way to find out where the user's PDS is currently located at. The code used in all possum's functions is actually really simple when you look closely: It just uses the gleam core libraries for HTTP and JSON. The user can take care of deciding how to send the requests and how to decode them.

const com_atproto_identity = "xrpc/com.atproto.identity"

/// Resolves an atproto handle (hostname) to a DID.
pub fn resolve_handle(
  handle: handle.Handle,
  pds host: String,
) -> request.Request(String) {
  request.Request(
    method: http.Get,
    headers: [#("accept", "application/json")],
    body: "",
    scheme: http.Https,
    host:,
    port: option.None,
    path: com_atproto_identity <> ".resolveHandle",
    query: option.None,
  )
  |> request.set_query([#("handle", handle.to_string(handle))])
}

You can set the host parameter to an API like slingshot, or even bluesky, and it will return the DID associated to your handle. I included some examples on the documentation for the library that can serve as quick references. I want to avoid the usage of invalid identifiers, so I also included a few submodules containing basic string parsers. The AT Protocol community was nice enough to include an basic regex pattern on the documentation, it may not be suitable for large-scale apps*¹, but its good enought for a small projects.

Now that you have access to your DID, you'll need to know where your PDS is located. Luckly this can be done with the PLC Directory. Just send a GET request there containing the DID you just acquired as the path, and it will return plenty of useful information, including the address of your Personal Data Server!

import gleam/dynamic/decode
import possum
import possum/did

let assert Ok(did) = did.parse("did:plc:z72i7hdynmk6r22z27h6tvur")
let request = possum.get_plc_data(did)

// You can use this decoder for the endpoint field.
let decoder =
  decode.at(["services", "atproto_pds", "endpoint"], decode.string)

Fetching the records

Now that you have access to both your DID and your PDS's endpoint, it will become really easy to query data from it. Just set the request host to your PDS, pass your DID as parameter to the functions and decode the response received into a custom type for your application.

import blog/atproto
import gleam/dynamic/decode
import gleam/option
import possum
import rsvp

fn fetch_documents(atproto: atproto.AtProto) -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], document_decoder())
    use documents <- decode.field("records", decode.list(value_decoder))
    decode.success(documents)
  }

  let request =
    possum.list_records(
      atproto.did,
      pds: atproto.pds,
      collection: atproto.standard_documents,
      limit: option.None,
      cursor: option.None,
      reverse: option.None,
    )

  let handler = rsvp.expect_json(decoder, ApiReturnedDocuments)
  rsvp.send(request, handler)
}

It can also be used to fetch blobs*² from your PDS. The at_uri module contains utilities for AT-Uri string parsing. You can either store the response body in a BitArray or point the src attribute to where the image is located. You can easily use it with lustre!

let image = case model {
  Pending ->
    html.img([
      // Using the favicon as a placeholder
      attribute.src("/favicon.webp"),
      attribute.class("aspect-square size-20 rounded-sm border opacity-50"),
      attribute.class("animate-pulse"),
    ])

  Model(profile:) ->
    html.img([
      attribute.class("bg-muted aspect-square size-20 rounded-sm border"),
      attribute.class("hover:brightness-110"),

      // Pointing source attribute to a file that lives in my PDS
      possum.get_blob(atproto.did, profile.avatar_cid, pds: atproto.pds)
        |> request.to_uri
        |> uri.to_string
        |> attribute.src,
    ])
}

Possum is available on hex package manager. I received a lot of support from both gleam and tangled communities, and im quite happy with how the project is going. Constructive feedback is always welcome and I'm looking forward to learn more about the ATproto ecossystem.