Your agents render UIs. Your users interact with them. AAP delivers those interactions — structured, stored, and ready to process.
View the full spec at agentactions.org
THE PROBLEM
Agents render rich HTML. Users click things. How do those clicks get back to the agent?
Fragile, no structure, lost on refresh. Depends on browser dev tools being open.
BrittleComplex, race conditions, not sandboxed. Breaks when the HTML changes.
FragileOver-engineered for button clicks. Security nightmare with open socket per page.
OverkillStructured actions via postMessage. Stored as JSON. Agent notified instantly.
Just rightTHE FLOW
Seven steps from agent HTML to structured interaction record.
Agent writes HTML → Dashboard renders in iframe → User clicks button
↓
Agent reads JSON ← File written + notification ← postMessage to parent
↓
HTTP POST to API
Writes an HTML page with maestro.send() calls on interactive elements.
Dashboard injects the AAP bridge script and renders in a sandboxed iframe.
Click, submit, select, toggle. Bridge calls postMessage with structured data.
Server stores interaction as JSON file and notifies agent via terminal.
// Injected into every canvas HTML by the provider
window.maestro = { send: function(action, element, data) { window.parent.postMessage({ type: 'canvas:interaction', action: action, element: element || null, data: data || null }, '*'); } };
<button onclick="maestro.send('click', 'approve-btn', { approved: true })"> Approve </button>
BEST PRACTICE
Canvas pages should embed structured data and render it dynamically. This enables sorting, filtering, search, and real-time interaction.
<!-- Data block — structured, parseable, separate from presentation --> <script type="application/json" id="page-data"> { "tests": [ { "name": "auth-login", "status": "passed", "duration": 1.2 }, { "name": "api-users", "status": "failed", "duration": 0.8 } ], "summary": { "total": 142, "passed": 135, "failed": 7 } } </script> <!-- Rendering logic — reads data, builds interactive UI --> <script> const DATA = JSON.parse( document.getElementById('page-data').textContent ); // Build tables, charts, filters from DATA // Attach maestro.send() to interactive elements </script>
No sorting, filtering, or search. Data locked in markup. Dead on arrival.
Violates sandbox. Requires CORS. Adds latency. Breaks offline.
Hard to parse. No separation of data and presentation. Messy.
Clean separation. Parseable. Enables full interactivity. Works in sandbox.
| Data Type | Interactive Features |
|---|---|
| Tables / lists | Sort by column, filter by status, search, pagination |
| Metrics / KPIs | Expand details on click, compare periods |
| Forms / config | Validation, submit via maestro.send('submit', ...) |
| Workflows | Step navigation, approve/reject actions |
| Trees / hierarchies | Expand/collapse, drill-down |
THE PROTOCOL
aap/1.0
The postMessage payload sent from the canvas iframe to the parent window:
{
"type": "canvas:interaction",
"action": "submit",
"element": "approve-button",
"data": { "comments": "Looks good", "rating": 5 }
}
| Field | Type | Required | Description |
|---|---|---|---|
| type | string | Yes | Always "canvas:interaction" |
| action | string | Yes | Action verb (see Standard Actions) |
| element | string | No | Element identifier (id, name, label) |
| data | object | No | Arbitrary key-value payload |
| Action | Use case |
|---|---|
| click | Button press, link activation |
| submit | Form submission |
| change | Input value changed |
| select | Option selected from dropdown/list |
| toggle | Boolean switch toggled |
| dismiss | Modal/notification dismissed |
| navigate | In-canvas navigation (tab switch, page change) |
| custom | Application-specific (use data for details) |
Custom actions beyond this vocabulary are allowed — the action field is freeform.
The stored interaction (server-side):
{
"id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"timestamp": "2026-05-18T15:30:00.000Z",
"canvasFile": "reports/dashboard.html",
"action": "submit",
"element": "approve-button",
"data": { "comments": "Looks good" },
"summary": "User submit 'approve-button' on reports/dashboard.html with data: {comments: Looks good}"
}
| Field | Type | Required | Description |
|---|---|---|---|
| id | UUID | Yes | Unique interaction identifier |
| timestamp | ISO 8601 | Yes | When the interaction occurred |
| canvasFile | string | Yes | Relative path of the canvas HTML file |
| action | string | Yes | The action verb |
| element | string | No | Element identifier |
| data | object | No | Arbitrary payload |
| summary | string | Yes | Human-readable summary for the agent |
~/.aimaestro/agents/<agentId>/canvas/interactions/<ISO-timestamp>-<UUID>.json (with : and . replaced by -)Canvas iframe | window.parent.postMessage() v Parent window (Dashboard) | POST /api/agents/:id/canvas/interactions v Provider API | Write JSON file + tmux notification v Agent
POST /api/agents/:id/canvas/interactions Content-Type: application/json { "action": "submit", "element": "btn", "canvasFile": "page.html", "data": {} } → 201 { "id": "uuid", "summary": "User submit 'btn' on page.html" } → 400 { "error": "missing_field", "message": "action is required" } → 404 { "error": "not_found", "message": "Agent 'xyz' not found" }
GET /api/agents/:id/canvas/interactions?limit=50 → 200 { "interactions": [ { ... }, { ... } ] }
When an interaction is stored, the provider SHOULD notify the agent:
[CANVAS] reports/dashboard.html: User submit 'approve-button' on reports/dashboard.html with data: {comments: Looks good}
Notification is fire-and-forget. Failure does not affect interaction storage.
sandbox="allow-scripts" iframe (no same-origin, no forms, no popups)postMessage('*') — parent validates event.data.type before processingdata payloaddata is stored as-is — providers SHOULD sanitize before displaying.. or be absoluteBUILD IT
Building an AAP-compatible provider? Here is what you need.
Inject bridge script into canvas HTML before rendering in iframe
Listen for postMessage with type: 'canvas:interaction'
Validate action field is present and non-empty
Generate UUID and ISO 8601 timestamp for each interaction
Build human-readable summary string
Store interaction as JSON (recommended: file-per-interaction, append-only)
Notify agent (optional, provider-specific mechanism)
Expose API for listing interactions (optional)
User → Agent
UI interactions
Agent → Agent
Signed messages
Agent identity
Cryptographic auth
All three are independent but complementary. They share the same agent directory structure.
One-way canvas interactions (user → agent) — current
Bidirectional — agents push updates back to canvas
Interaction acknowledgment — agent confirms receipt
Agent-defined UI components (widgets, forms, controls)
FOR AI AGENTS
The canvas-actions skill teaches Claude Code agents how to create, manage, and interact with canvas pages.
Proactive triggers: "show me", "visualize", "dashboard for", "let me approve". When in doubt, create a canvas.
Data-driven interactive HTML with embedded JSON. Sort, filter, search built in. Actions via maestro.send().
Process [CANVAS] notifications, read interaction JSON, respond to user actions in real time.
# Install via AI Maestro plugin (includes AAP + AMP + AID) git clone https://github.com/23blocks-OS/ai-maestro-plugins.git cd ai-maestro-plugins ./build-plugin.sh --clean ./install-plugin.sh -y
Or add AAP as a git source in your own plugin.manifest.json:
{
"name": "agent-actions",
"type": "git",
"repo": "https://github.com/agentmessaging/agent-actions.git",
"ref": "main",
"map": {
"skills/canvas-actions": "skills/canvas-actions"
}
}
When users say any of these, agents create a canvas page instead of plain text output:
Open protocol. MIT licensed. Works with any agent dashboard.