What building Shippy taught us about building agents

Shippy is a maritime AI agent built for high-stakes decisions, where the wrong answer has real impacts. Here's the architecture behind it—and the lessons we're carrying into Ai2's other environmental platforms.

Shippy answering a live query about Ghana's EEZ. The response shows its work: the boundary source, the data cutoff, the query timestamp, and a deep link back to the Skylight map so the analyst can verify every number.

Building an AI agent for a high-stakes operational domain like protecting the ocean is, above all, a problem of reliability. For a maritime analyst, a wrong answer could send a patrol vessel miles in the wrong direction, costing significant resources that are already stretched thin and potentially putting personnel in harm's way.

So when the Skylight team set out to build Shippy, our AI for real-time maritime domain awareness, the real work wasn't the model. It was building a system we could trust to be correct, to stay within its limits, and to hold up across a wide range of tasks. And we had to verify all of it against Skylight's live data, updated continuously as new satellite and vessel signals arrive—not a static snapshot.

We think of an agent like Shippy as three things: a soul, skills, and config.

The soul is the system prompt that frames Shippy's persona and sets behavioral boundaries. Skills tell Shippy how to handle specific kinds of requests. Together, the soul and skills are baked into a Docker image—a versioned, deployable artifact that defines what Shippy is. Config covers everything else: which agent harness to run (in Shippy’s case, OpenClaw, an open-source agent framework), which LLM to use (currently, Shippy relies on Claude Opus 4.6), and runtime settings. Secrets like API keys are injected at runtime; swapping the model or the harness is a config change, not a rebuild.

Shippy’s skills follow the same agent-skills spec used by coding tools like Claude Code and Codex—plain markdown files with structured frontmatter. This keeps each skill comprehensible, versioned, and easy to revise. Shippy currently includes skills for:

Querying the Skylight API for Events (the vessel behaviors Skylight surfaces, like fishing or transshipment between two vessels) and vessel data

Looking up Exclusive Economic Zones (EEZ) and Marine Protected Area (MPA) boundaries

Interpreting vessel track data, the position and movement signals ships broadcast, building on the activity classifications Skylight's models, including Atlantes, already produce

Generating interactive map links that let an analyst jump from a Shippy in-chat answer to an exact location on the Skylight map

For example, the Skylight API query skill encodes the full workflow for answering a question about a specific area. When an analyst or user asks, "show me fishing activity in Panama's EEZ last month," the skill's instructions direct Shippy to first resolve "Panama EEZ" to a boundary polygon through Skylight's regions API instead of guessing or hard-coding coordinates, then query Fishing Events within that geometry, format the results with deep links back to the Skylight map, and attribute any vessel metadata drawn from Skylight partners like Global Fishing Watch or TMT.

A single question posed to Shippy can hook into several skills at once. "Are there vessels operating near the Cordillera de Coiba MPA?" draws on the Skylight skill for data query, our partner ProtectedSeas’ database for MPA boundary context, and the vessel track skill for interpreting vessel behavior. All of this happens in a single dialogue turn.

The soul defines what Shippy will and won't do. It won't make legal determinations about whether a vessel is breaking the law—that is a determination for people, not an agent. It also won't speculate beyond what the data supports. These boundaries are explicit in the system prompt, not implicit in fine-tuning, which makes them auditable and easy to revise.

Agents are nondeterministic. You can't control what the model decides to do, but you can make the tools it reaches for predictable. To that end, Shippy 'talks' to Skylight through a purpose-built CLI that calls the API, rather than issuing raw calls itself.

Our API has dozens of input types, nested filter objects, pagination cursors, and complex geometry inputs. In early prototypes, we let Shippy construct API calls from scratch. It produced a steady stream of subtle bugs: malformed pagination that silently dropped results, geometry encoding errors, and correct-looking queries that returned wrong data because of a misunderstood filter type.

The Skylight CLI collapses that complexity into a predictable interface. Shippy issues a single command – skylight events search with typed filter flags – and the CLI handles authentication, pagination, and structured output. The CLI is also self-documenting: extensive --help text and detailed error messages give the agent (and human developers) enough context to recover from mistakes without guessing. Its output is always written to a local JSON file rather than piped through the shell. Early on, large result sets would hit pipe buffer limits or break downstream tools like jq. Writing to disk sidesteps both problems and lets the agent programmatically access query results across subsequent steps.

Underneath the CLI is a standardized API: multiple resource types – Skylight Events, vessels, regions, satellite imagery, vessel tracks, and more – accessible through a common pair of operations, search and aggregate. The APIs' inputs and outputs are defined as typed schemas with field-level descriptions.

This layering – typed API, deterministic CLI, and agent skills that reference the CLI's commands – means that each of Shippy's components can be tested independently. The API has its own test suite. The CLI can be exercised by a human or an agent. And the agent skills reference CLI commands that handle the plumbing so that Shippy doesn't have to reinvent the wheel every time it hits the Skylight API. Each layer narrows what the next layer can get wrong.

Skylight serves hundreds of government agencies and NGOs across over 70 countries. A fisheries officer in the Philippines has Areas of Interest, vessel watchlists, and alert configurations that are scoped to their Skylight account. When they ask Shippy a question, the agent's API calls need to return their data, and their conversation history must never be visible to anyone else.

Every user talks to Shippy inside their own ephemeral, isolated session, and making that work reliably at scale was one of the most significant engineering efforts behind the project. We built Mothership, an agent hosting platform that provisions a dedicated Kubernetes deployment for each user session. When a user opens a conversation, the system spins up a set of pods packaging the agent runtime, its skills, and the Skylight CLI. The user's Skylight JWT is injected at provision time so the agent's API calls are scoped to that user's data.

Files the agent writes during a multi-step analysis exist only within that session and are never shared across users. Inside the sandbox, the agent can write and run code, install dependencies, pull in datasets, and work through multi-step analyses. At the network level, the sandbox is restricted to only the services it needs.

Most benchmarks rank general-purpose AI on static questions. They don't capture how an agent behaves once it's wired into a real workflow: how it selects tools, queries live data, acts on results, and knows where to stop. So we built our own eval system around how Shippy works, scoring the whole agent – model, skills, and sandbox together – against live data.

In our eval framework, subject-matter experts write scenarios and rubrics, choosing which criteria apply to each task and setting the weights, so every task is graded on what actually matters for it. A fishing-events query, for instance, weights data accuracy m