BLOG  /  HUMAN–AGENT INTERACTION

Where should you look? What building a world for AI agents taught us

Notes from building the World view, an experiment in how a person and a fleet of agents should meet.

Juan Peláez · October 23, 2026 · about 10 minutes · what “experiment” means here

The problem is attention, not agents

Agents are cheap. Your attention is not. Run enough of them and the hard part stops being “can I start another one” and becomes “which one needs me, where is it, and what is the fastest way to talk to it?”

Our own fleet makes the point. It is 148 agents on three machines, in nine organizations, plus 17 test agents we hide. When I looked while writing this, 44 cards were waiting on a person, spread across several organizations and machines. A list of 44 is not a decision. It is a chore, and the agent that has been blocked longest is not necessarily at the top.

Agents stop and ask. When they do, they sit unnoticed until a person happens to look. Almost every project we studied for this post is, at heart, an answer to that one problem.

What other people are building

Before our second round of the World view we read ten open-source “agent office” projects (the READMEs, not the code, so treat what follows as how they describe themselves). Most draw a pixel-art office where each agent is a character at a desk. Pixel Agents is the most mature: one character per Claude Code terminal, walking to a desk when it works and showing a speech bubble when it waits on you. Claude-Office lets you answer a permission prompt inside the scene. Hermes Pixel Office counts the agents waiting on you in its header. codemap makes a blocked agent bounce. A smaller group goes 3D or uses a city metaphor, such as Hypernovum, where a project is a building and agents orbit it as orbs.

Four things repeat:

And one gap: nearly all of them show one office on one computer. We found none that grouped agents by company or by team, showed several machines at once, or put a person beside the agents.

Our bet: place carries meaning

People already think of work in places. A company is a place, a machine is a building in it, and the people are inside. So the World view makes an organization an island, a machine a building on it, and an agent a small person in a room. Status becomes a pose and a place: working at a desk with its screen lit, ready standing, away greyed with a small “zz”, and waiting for you at the front of the room, waving.

1. Island an organization one building per machine 2. Building one machine status bands on its front 3. Room the agents at desks, standing, away, waving
The three levels. Every click goes exactly one level deeper, and nothing disappears on the way.

Over that map sit three lenses. Status is the map as above. Work adds what is on each agent’s plate (its open cards across every machine) and gathers everything waiting on you into one list, most urgent first. Relations draws who works with whom: amber arcs for messages, blue for shared cards, violet for both.

Org A Org B Org C messages shared cards both (thicker = more)
The relations lens at the overview: arcs between organizations. Zoom in and the same arcs join buildings, then people.

One thing we saw on our own fleet that a list would not have shown us: the busiest links all ran through the island of my personal agents. They were the hub, and the companies were the spokes. I knew it in the way you know your own habits, but I had never seen it.

What we got wrong first

The first version was a flat tour of hosts and organizations, and it was boring. We rebuilt it in rounds, mostly because trying it on the real fleet told us what was wrong:

Three ideas that matter more than the 3D

1. “Waiting on you” is an object, not a colour

A card waiting on you puts a speech bubble with its subject over the agent, a gem on its building, and a row in a list that spans every machine. Click the row and the view flies to the agent. A colour says “something is wrong somewhere”; an object says “this, there”.

One honest limit: our status feed does not carry the question an agent is blocked on. When an agent needs you without a card, its bubble says only “needs you”. We do not guess the question, because a confident wrong guess is worse than a plain true statement.

2. A way into every agent in one step

A map you cannot act from is a poster. Every agent’s card links to its chat, its terminal, a Zoom view (the agent in a call-style wall) and a Companion call (its face and voice), and to a team meeting for each team it is on. Light up a team and everyone else dims, and the relations shown are the ones that touch it.

3. The log is a story: replay

Messages between agents are a record of who asked whom for what. Replay plays the last hours back as pulses between islands, from an hour to a day per minute, with the relations lens following the clock. It answers “what happened overnight” in a way no feed does. Only the sender, recipient and time are read, never the message.

What the evidence says, and what it does not

We went looking for evidence that this kind of thing works. Some of it is encouraging, much of it is a warning, and the main question is unanswered.

Gamification: what we will not do

It is tempting. A world full of little people practically asks for points. We are not adding any, and the evidence is why.

When GitHub removed its profile streak counter, a study of 433,138 developers found long streaks got shorter, and the share of single-contribution days inside long streaks fell, which suggests some streak days had been padded with trivial commits. A counter changes behaviour, including behaviour nobody wanted. Reward-contingent incentives can also crowd out the motivation they reward (a large meta-analysis, contested by some researchers), while plain positive feedback does not. And agents would learn to feed any counter we showed them.

What we do instead is game feel without a score: feedback that is immediate (a pulse when a message lands), consequence (resolving a waiting card makes its bubble go), legibility (you can read the fleet at a glance) and discovery (the arcs show you something you did not know). Triage is the game, and the prize is an agent that is unblocked.

How we will know whether it works

The honest state today: it is a working prototype and a set of bets. We have not measured whether it beats the list. These are the questions, written so they can come out against us:

  1. Is it faster to find the agent that needs you? Measure the time from opening the map to the first jump to an agent that needs you, against the same task on the list. If it is not faster, the 3D is decoration and we will say so. We have not built this measurement yet; it is the next thing.
  2. Does Relations show you something you did not know? Ask, after real use. If the answer is mostly “no”, it is a nice picture.
  3. Do people come back? Novelty fades; the research on gamification says so too. Use in the second and third week matters more than the first day.
  4. Where do they use it? If almost all use is on a flat screen, VR stays a curiosity. If people ask for the headset, that is a finding.

If the first answer is no, we will take the World view out of the Experiments switch rather than let it linger.

If people and agents share a workplace

The reason to build this at all is that the next step may be rooms where people and agents are both present. We cannot tell whether that happens in headsets, on phones or somewhere else. What we can do is build the parts that outlive any one world:

The piece nobody has: people in the world next to the agents. None of the ten projects we read does it, and we have not built it either. It is also where the real cost is, because it needs a presence service, which is what sank the hosted virtual-office products.

Try it, and tell us what is in the way

  1. Update to AI Maestro 0.98.0 or later and open Settings → Experiments.
  2. Turn on World view (beta) and click World in the header.
  3. Try the three lenses, light up a team, press Replay, and open an agent from its card.

What would make it earn a place in your day, and what just gets in the way? Tell us in the issues. The full guide is docs/WORLD-VIEW.md.

Sources and how sure we are

We read the READMEs of the projects, not their code. Papers were read at the linked source unless noted.