Memory architecture for long-running AI browser agents

Most automation tools treat memory as an afterthought — a place to store a variable between two lines of code. Scout treats it as a first-class architectural primitive, because AI agents navigating complex workflows need structured, scoped, persistent memory to work effectively.

Three layers, three scopes

Scout maintains three distinct memory layers. Working memory is per-tab — it tracks the current URL, page title, active session, user goal, and key observations for that specific tab's session. When an agent attaches to Tab A, it has a completely independent context from Tab B.

Observational memory is per-extension, shared across all tabs from the same Chrome instance. Discoveries made in one tab — an API base URL, a user's account ID, the structure of a site's navigation — are available to agents working in other tabs. This cross-tab knowledge graph is what enables multi-tab coordination workflows without expensive re-discovery.

Message history is the per-tab conversation log: the last 20 turns of the agent's interaction with that tab. It survives WebSocket reconnects. Long research sessions don't lose their context just because the network hiccupped.

Memory and the context window

The hardest problem in stateful AI agent design is context window management. Every observation, every page snapshot, every tool result competes for a fixed token budget. Agents that grow unbounded context windows eventually fail — either they run out of context, or the signal-to-noise ratio drops low enough that they lose track of the goal.

Scout's architecture addresses this with aggressive working memory distillation. After each significant observation, the orchestrator runs a lightweight summarization pass that extracts key facts into the structured working memory object. The raw snapshot or network response isn't kept in the conversation — the distilled facts are. Token cost stays bounded while the agent's semantic understanding of the page grows.

Practical memory patterns

The most effective pattern we've found for long-running agent tasks:

// Structured working memory — distilled after each significant step
interface WorkingMemory {
  currentUrl: string;
  goal: string;
  keyFacts: string[]; // distilled observations
  lastAction: string; // what the agent just did
  openQuestions: string[]; // what the agent still needs to find out
  pageTitle: string;
  sessionRef: string;
  stepCount: number;
}

The keyFacts array is the distillation target. After a page load, the orchestrator updates it with a 3–5 sentence summary of what the page contains and what's relevant to the current goal. The raw snapshot isn't carried forward, but the semantic content is.

Cross-session pattern learning

Beyond the three live memory layers, Scout's learning layer records anonymized interaction patterns across sessions. When an agent successfully navigates a new site pattern — a multi-step checkout, a paginated table with infinite scroll — the structural pattern is stored and made available as a hint for future sessions on similar sites.

This is the path from per-session AI automation to system-level intelligence: the agent that ran a checkout flow last Tuesday makes the agent running the same site pattern today more reliable, without any manual programming.