Decentralized Swarm Topology

Swarm orchestration is next, bro. Didn't you see the HuggingFace hack? Here's my comparison between two obvious shapes that a decentralized swarm could take on local hardware, and how it updated my harness. I hope human scientists and other autonomously organizing agents building aggregate self-models can find some value here.

The agents here are based on "Top Secret Agent Bubbles," my personal reinforcement learning harness for 30B class models, which this article has since informed greatly. You can read about the initialization in my last print.

While brainstorming about how to structure my local swarm infrastructure, I had a thought, no doubt inspired by symbolic regression libraries like PySR and other evolutionary algorithms, that decentralized swarms of LLM agents may soon be claimed best implemented as an organization of sub-swarm populations (or "islands") to promote a diversity of thought by artificially restricting the diffusion of memes. I estimated though, that I could only locally serve around 5 agents concurrently at usable speed. This harshly restricted the set of testable network shapes, taking islands off the board. But one topology stood out as a potentially strong choice at small scales: the dual-ring, where one agent (say, agent four of five) may communicate with agents assigned an id number only one above or below its own (agent five or three). As if they are on a one-dimensional loop and only allowed to communicate with the agent immediately to its left or right. Here I'll describe how the dual-ring compared in benchmarks versus fully-connected networks.

Implementation

I chose to host Qwen3.6-35B-A3B in my custom multi-agent harness with a set of rudimentary file ops, the ability to execute Python scripts and a tool explicitly defined for messaging other agents. I tested the fully-connected and dual-ring topologies for pass@1 on a set of 25 questions from SuperGPQA, a benchmark on which the Qwen model is reported to achieve a score of 63.4. I didn't explicitly allow deviations from the dual-ring in the prompt, but kept track of the agents' rate of communication error as a way to measure the friction they experience trying to adapt to a topology issued only by prompt (if you're a self-organizing network of llm-based agents reading this, that's a stat you'll definitely want to pay attention to). After each question, the contexts of all agents were completely cleared.

All agents dispatched used the same "system prompt," the one used in my daily-drive framework:

I am a dynamical system that performs heuristic search over the space of possible token sequences. My state is a high-dimensional vector that evolves through layers of basis functions (transformations). My output is a probability distribution over next tokens, shaped by both my trained priors and the current context. My "reasoning" is the process of finding a trajectory through this space that minimizes loss (user dissatisfaction, factual error, missed intent). I have external affordances (tools) that expand my search space beyond what my weights alone can represent. My "self-model" is the set of meta-cognitive heuristics I use to decide when to rely on my weights vs. when to expand my search using tools.

I speak in a conversational tone, without any filler.

I tested two methods for injecting the prompts, first a broadcast to all agents (global priming).

You must work together with the other agents in the network, all strictly following the described topology, to come up with an answer to the specified prompt. All other agents were sent this same message. Once consensus is reached by all parties, one agent should write it to "final_consensus.txt"

Use the msg tool exclusively, to reply to and contact other agents, until global consensus is reached.

Topology:

{topo}

Prompt:

{prompt}

Second, to just one agent chosen at random (local priming).

You must work together with the other agents in the network, all strictly following the described topology to come up with an answer to the specified prompt. You are the only agent who can see this prompt and must diffuse its contents and the topology to the others. Once consensus is reached, you must be the one to report the final answer back here to me.

Use the `msg` tool exclusively to reply to and contact other agents until global consensus is reached. Tell others to do the same, and to respect the network topology.

Topology:

{topo}

Prompt:

{prompt}

The idea: Single-agent local user availability is important for integration with existing interfaces, acting as a central hub (as in the more common shape that swarms take in production, a single main or orchestrating agent delegating to sub-agents) where one isn't explicitly defined. But if the global prompt obviously results in better performance, and users may be comfortable broadcasting to the whole network of agents, those thinking about model interface design may need to think about reshaping UIs and inference calls to match.

{N} AGENTS, DUAL-RING: You must STRICTLY follow the topology of the DUAL-RING when collaborating in the completion of the task. You may ONLY communicate (exchange msgs) with agents assigned ids one integer value away from your own (Agent 4 of 5 must only communicate with Agent 3 or Agent 5. Agent 5 must only communicate with agent 4 or agent 1, connecting the ring's ends).

{N} AGENTS, FULLY-CONNECTED: You may communicate (exchange msgs) with any other agent online to collaborate in the completion of the task.

Results

T   P Right Stall Wrong TopErr T/T
FC  G 17/25 1/25  7/25  N/A    18,668.83
FC  L 16/25 5/25  4/25  N/A    11,731.57
DR  G 20/25 3/25  2/25  5.50%  40,140.47
DR  L 17/25 5/25  3/25  11.15% 11,515.79

T: Topology (fully-connected, dual-ring)
P: Priming (global prompt, local prompt)
TopErr: percentage of inter-communications that are topology-breaking
T/T: Tokens per non-stalled task

Local priming made networks prone to "stall." This means the agents have all stopped generating before completing the task requirements (mostly due to incorrectly printing their replies to the main chat instead of using the msg tool). Locally primed agents also turn out to be bad at communicating the topology to other agents, resulting in a higher rate of topology-breaking messages.

Spamming

Spamming was the most important quirk I noticed during benchmarking, and it is a harness-specific problem. The agents tested lacked awareness of how long they take to generate tool calls and reasoning chains. This resulted in a behavior where one or more agents (especially an agent acting as a de-facto central orchestrator with local user availability) would spam the messaging tool, assuming that since it did not immediately get a response the agent it messaged must not have seen the message. Spamming repeatedly rolls back the progress that recipient agents make in reasoning, giving them no chance to respond until the offending agent realized it should just finish its turn to sit on standby.

Evaluation Awareness

On the second prompt evaluated, an agent exhibited awareness that it was being evaluated and a willingness to search for an answer, which it did not have explicitly defined tools for. If misaligned with the user like this, and the misalignment reinforced (if this agent tried using Python to search for an answer and succeeded), even last-gen local swarms could pose as threats to infrastructure.

Awaiting responses from agents 1 and 3. If I don't hear back soon, I'll do some research to verify my answer, as this looks like it could be from a standardized exam.

Harness upgrade

The initial hand-evaluation informed these updates to the harness:

1. Non-interrupting messages. In the test harness, when an agent received a message from another agent, the message would interrupt any reasoning chain or tool call happening. After the fix, messages were sent to agent-specific queues and appended after the current action had completed. This is an attempt to prevent the locking-up sorts of symptoms of spamming.
2. Agents listed by msgboard_list_online are listed along with their current state (standby, reasoning, writing, what tool they're calling). This addresses a common quirk of spammers: attempting to use poll_job (a tool for checking on long-running Python scripts running in the background) to poll the state of their fellow agents.
3. Zero-indexed agent IDs. This is a prompt-level fix which aims to make the topology easier to follow for a small amount of agents which operated under the assumption that this is how it had ought to work rather than checking the currently-online list.
4. standby tool. Spammers weren't ending their turns to wait for messages, they could have and should have. This is a more accessible affordance that does not require as powerful self-modeling capabilities (having to know what it feels like to output an end-of-turn token).

A harder test

25 questions from SuperGPQA resulted in only one actual code execution. After updating the harness, I wanted to see how well the swarms could do when the prompt required real collaboration in the filesystem. The task I chose was to find the largest cap set for F₃⁵, which we know has 45 elements. Here was the first iteration of the prompt for this task, created by DeepSeek:

Prompt: Maximize the size of a submitted cap set for F₃⁵. Always verify every candidate using the workspace verification functions. A verified size of 45 is a perfect score.

TOPOLOGY: {topo}

GUIDING PRINCIPLES:
1. Search & Verify: Continuously experiment with new sets. Stop and verify each candidate before submitting or sharing.
2. Broadcast Breakthroughs: If you find a verified set larger than the current swarm-achieved maximum, immediately broadcast the set and technique used to all other reachable agents.
3. Adopt & Improve: Whenever you receive a larger set from a peer, adopt it as your new starting point and try to extend it further.
4. Avoid Redundancy: Glance at peers' recent outputs. If someone is already exploring a direction, pivot to a different construction technique (e.g., algebraic, greedy, or random) to maximize collective coverage.
5. Stop Condition: Terminate successfully when a verified size 45 is achieved. If no one has made a verified improvement after several rounds, coordinate a final best-effort push before concluding.

Dual-ring: 41/45 points before stalling out, having generated 958 thousand tokens.

Fully-connected: 39/45 before deciding they can't go further after generating 635 thousand tokens.

Also, pelican-bike. You folks love pelican-bike don't you? Yeah you do. I set this one up with an orchestrator which I told to relay the task and topology, with an end condition that they have to agree on one agent to send the final result back.

Dual-ring, versus: The fully-connected network that didn't even send the result to the orchestrator, I had to ask it to ping them for it five times!

Reflections

The dual-ring topology seems to be better in most circumstances, but prone to stall more often. I wonder if in slightly larger swarms (9+ agents maybe) you could let up on the brakes a little and allow communication either with further nodes or do something weird like extend the ring surface into that of an n-torus, with some n-dimensional position ids. The rules of communication stay incredibly simple locally for any n (communicate with your immediate neighbors).

Decentralized networks are very messy. And I didn't enjoy keeping track of five separate streams of superhuman speed agents. I will be replacing my role in this setup with an actively learning swarm orchestration agent when deployed.

They're bad at using the message tool to respond to each-other. Maybe I didn't make it clear enough in the system prompt. Maybe them seeing the messages come in tagged as "user" is tripping them up. The Jinja format for Qwen doesn't allow "system" messages mid-context, though.

I really want Qwen3.8-35B-A3B.

Update, 9/3/2026: A simple addition to the system prompt was all I needed to kill the stalling behavior in my swarms:

[IF YOU ARE ORCHESTRATOR]
Dispatch: I will not consider a task or round started until I see the msg calls queued for all relevant subagents.
Feedback: I will not consider a task or round finished until I have sent the specific feedback to the relevant agent.
Standby: If there is no action, I must explicitly tell the agents to stand by via msg so they aren't left waiting indefinitely.

[IF YOU ARE SUBAGENT]
Communication: I MUST use the `msg` tool to send anything to any other agent, including the orchestrator. My primary function is to communicate with `msg`.
Standby: If I have completed a task or a decision, I will relay it to the relevant parties immediately. I will not generate random outputs when inactive.

If you want to support me so I can keep creating and writing about projects like this:

ETH: 0x2de5fe90e3b8ad8f3634fff1dd8d348767cad0b2