View on GitHub

Lightspeed Core Stack

Lightspeed Core Stack

Feature design for Widget System (AG-UI Protocol)

   
Date 2026-10-05
Component Core / Endpoints / Configuration
Authors Chih-Tao Lee
Feature LCORE-4575
Spike LCORE-4288
Links AG-UI Protocol, pydantic-ai AG-UI

What

A new /agui endpoint that speaks the AG-UI (Agent-User Interaction) protocol, enabling agents to drive rich, interactive UI widgets (forms, confirmation dialogs, data tables, charts) in customer frontends. The endpoint uses pydantic-ai’s built-in AG-UI integration (AGUIAdapter) to translate agent execution into AG-UI protocol events streamed as SSE.

Customers define UI-related tools on their own MCP servers. The AG-UI protocol handles structured delivery of tool calls, state synchronization, and rendering data to the frontend. All existing LCS cross-cutting concerns — authentication, authorization, BYOK, MCP, conversations, conversation compaction, shields, quota, and telemetry — are integrated into the new endpoint.

Key capabilities:

Why

Today, Lightspeed Stack has limited options for agent-driven UI:

This creates problems as more teams evaluate migrating to Lightspeed Stack:

The AG-UI protocol solves these by providing a standardized, widely adopted event vocabulary that any compatible frontend can consume. The immediate driver is preparing for a potential PCM team migration, but the design accommodates any team needing agent-driven UI in the future.

Requirements

Use Cases

Architecture

Overview

                                                    ┌──────────────────────────────┐
┌─────────────────────┐   POST /agui                │       Lightspeed Stack       │
│                     │   (RunAgentInput)           │                              │
│   Frontend App      │ ─────────────────────────▶  │  ┌────────────────────────┐  │
│   (CopilotKit /     │                             │  │    /agui endpoint      │  │
│    custom client)   │  ◀───────────────────────── │  │                        │  │
│                     │   SSE (AG-UI events)        │  │ ┌──────────────────┐   │  │
│  ┌───────────────┐  │     RunStarted              │  │ │  AGUIAdapter     │   │  │
│  │ Widget        │  │     TextMessageContent      │  │ │       │          │   │  │
│  │ Renderer      │  │     ToolCallStart/Args/End  │  │ │       ▼          │   │  │
│  │  (forms,      │  │     StateSnapshot/Delta     │  │ │  pydantic-ai     │   │  │
│  │   dialogs,    │  │     RunFinished             │  │ │  Agent           │   │  │
│  │   charts)     │  │                             │  │ │       │          │   │  │
│  └───────────────┘  │                             │  │ │       ▼          │   │  │
│                     │                             │  │ │  OGX (LLM)       │   │  │
└─────────────────────┘                             │  │ └──────────────────┘   │  │
                                                    │  └───────────┬────────────┘  │
                                                    │              │               │
                                                    │              ▼               │
                                                    │  ┌────────────────────────┐  │
                                                    │  │ Customer MCP Server    │  │
                                                    │  │ (UI tool definitions)  │  │
                                                    │  └────────────────────────┘  │
                                                    └──────────────────────────────┘

Request flow

  ┌─────────────────────┐
  │ POST /agui          │  RunAgentInput (AG-UI protocol request)
  │ (RunAgentInput)     │  Contains: messages, state, frontend tools, threadId
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────┐
  │ Auth + Authorize    │  get_auth_dependency() + @authorize(Action.AGUI)
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────┐
  │ Parse AG-UI input   │  AGUIAdapter.from_request() → RunAgentInput
  │                     │  Extract threadId → conversation_id
  │                     │  Determine conversation mode (client / server)
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────┐
  │ LCS middleware      │  check_mcp_auth()
  │                     │  check_tokens_available()
  │                     │  validate_model_provider_override()
  │                     │  Azure Entra ID token refresh (if BYOK)
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────────────────────────────────────────────────┐
  │ Conversation mode                                               │
  │                                                                 │
  │  ┌─────────── client ───────────┐  ┌──────── server ─────────┐  │
  │  │ Use full message history     │  │ Load persisted history  │  │
  │  │ from RunAgentInput as-is.    │  │ from LCS conversation   │  │
  │  │ No server-side persistence.  │  │ store. Append new items │  │
  │  │ No compaction.               │  │ from RunAgentInput.     │  │
  │  │                              │  │ Apply compaction if     │  │
  │  │                              │  │ approaching ctx limit.  │  │
  │  └──────────────────────────────┘  └─────────────────────────┘  │
  └──────────────────┬──────────────────────────────────────────────┘
                     │
                     ▼
  ┌─────────────────────┐
  │ Build + run agent   │  build_agent() with MCP tools + frontend tools
  │                     │  AGUIAdapter.run_stream()
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────┐
  │ Stream AG-UI events │  SSE response: RunStarted, TextMessage*,
  │                     │  ToolCall*, StateSnapshot/Delta, RunFinished
  └──────────┬──────────┘
             │
             ▼
  ┌─────────────────────┐
  │ Post-stream         │  Server mode: persist conversation turn
  │                     │  Emit OTEL telemetry
  │                     │  Consume quota tokens
  └─────────────────────┘

Endpoint implementation

The /agui endpoint lives in src/app/endpoints/agui.py. It follows the A2A endpoint pattern — a protocol adapter that reuses LCS utility functions for all cross-cutting concerns.

"""Handler for AG-UI (Agent-User Interaction) protocol endpoint."""

from typing import Annotated

from fastapi import APIRouter, Depends, Request
from pydantic_ai.ui.ag_ui import AGUIAdapter
from starlette.responses import Response

from authentication import get_auth_dependency
from authentication.interface import AuthTuple
from authorization.middleware import authorize
from models.config import Action
from utils.mcp.mcp_headers import McpHeaders, mcp_headers_dependency

router = APIRouter(tags=["agui"])


@router.post(
    "/agui",
    summary="AG-UI Protocol Endpoint",
)
@authorize(Action.AGUI)
async def agui_endpoint_handler(
    request: Request,
    auth: Annotated[AuthTuple, Depends(get_auth_dependency())],
    mcp_headers: McpHeaders = Depends(mcp_headers_dependency),
) -> Response:
    """Handle AG-UI protocol requests.

    Receives an AG-UI RunAgentInput, applies LCS middleware (auth, MCP,
    shields, quota, BYOK, conversation context, compaction), runs the
    pydantic-ai agent via AGUIAdapter, and streams AG-UI events as SSE.
    """
    # 1. Parse AG-UI request, determine conversation mode (client/server)
    # 2. Apply LCS middleware (auth, MCP, quota, BYOK)
    # 3. Resolve conversation context:
    #    - Client mode: use messages from RunAgentInput as-is
    #    - Server mode: load persisted history, append new items
    # 4. Apply compaction if needed (server mode only)
    # 5. Build agent via build_agent() + prepare_responses_params()
    #    (shield moderation via agent capability)
    # 6. Run via AGUIAdapter and stream response
    # 7. Post-stream: persist turn (server mode), emit telemetry, consume quota
    ...

Note: The endpoint handler accepts a raw Request rather than a typed Pydantic model because AGUIAdapter.from_request() parses the AG-UI RunAgentInput from the request body internally. LCS middleware is inserted between parsing and agent execution.

Authorization

Add AGUI to the Action enum in src/models/config.py:

class Action(StrEnum):
    # ... existing actions ...

    # Access the streaming query endpoint
    STREAMING_QUERY = "streaming_query"

    # Access the AG-UI protocol endpoint
    AGUI = "agui"

Router registration

Register in src/app/routers.py without a version prefix, following the A2A pattern:

from app.endpoints import (
    agui,
    # ... existing imports ...
)

def include_routers(app: FastAPI) -> None:
    # ... existing routers ...

    # AG-UI (Agent-User Interaction) protocol endpoint
    app.include_router(agui.router)

Conversation modes

The /agui endpoint supports two conversation modes — client and server — selected per-request. The mode determines who owns the conversation history and which LCS features are available.

Client mode (default)

In client mode the frontend owns the conversation history. The full message history is included in every RunAgentInput request, and LCS treats it as the complete context for the agent run. This aligns with the AG-UI protocol’s design where the client is the source of truth.

Server mode

In server mode LCS manages conversation persistence. The frontend sends only the new message items for the current turn, and LCS loads the existing conversation history from its conversation store, appends the new items, and passes the assembled history to the agent.

Conversation ID mapping

AG-UI threadId maps directly to LCS conversation_id via normalize_conversation_id(). This is a direct mapping — no intermediate context store is needed (unlike A2A which uses A2AContextStore because A2A contextId has different semantics).

adapter = await AGUIAdapter.from_request(request, agent=agent)
conversation_id = normalize_conversation_id(adapter.conversation_id)

The frontend always provides a threadId with every request. In server mode, the endpoint looks up whether the threadId already exists in the server conversation store — if it does, the existing conversation history is loaded; if not, a new conversation is created.

MCP tool integration

MCP tools are resolved through the existing mechanism. The endpoint calls prepare_responses_params() which internally resolves MCP tools via resolve_tool_choice(). These tools are passed to build_agent() and become available to the pydantic-ai agent alongside any frontend tools provided in the AG-UI RunAgentInput.

Frontend tools from RunAgentInput are handled by AGUIAdapter — they are converted to pydantic-ai tools and merged with the server-side MCP tools automatically. When the agent invokes a frontend tool, AGUIAdapter emits ToolCallStart/ToolCallArgs/ToolCallEnd events; the frontend executes the tool and submits results in the next request.

State management

AG-UI state is passed through without LCS interpretation (Decision T3 from spike). Customers manage state entirely between their MCP server tools and their frontend.

The AGUIAdapter handles state automatically:

LCS does not populate, validate, or interpret the state. Conversation metadata (ID, model, quota) is conveyed through other mechanisms (AG-UI lifecycle events, response headers).

Limitation: AG-UI state is not forwarded to MCP servers

AG-UI state and MCP tools live in different layers. State is available to local pydantic-ai tools (Python functions decorated with @agent.tool) via ctx.deps.state, but MCP tools are remote — they are invoked via the MCP protocol, which only transmits tool_name + arguments. There is no side-channel to pass AG-UI state to an MCP server automatically.

Customers who need their MCP-hosted UI tools to access AG-UI state have two options:

  1. Design MCP tool arguments to include needed state. The LLM sees the AG-UI state (it is part of the agent context) and can pass relevant parts as tool call arguments. This relies on the LLM correctly forwarding state fields, which can be guided via system prompt instructions or tool parameter descriptions.

  2. Use AG-UI frontend tools instead of MCP tools. For tools that primarily read or write UI state (e.g., updating a form, toggling a panel), define them as AG-UI frontend tools (executed in the browser) rather than MCP tools. Frontend tools have direct access to the shared state via the AG-UI protocol — no forwarding needed.

Note: a local pydantic-ai tool bridge (reading ctx.deps.state and calling the MCP server programmatically) is not feasible in LCS’s architecture. Customers provide tools via MCP servers — they do not control agent construction or register local Python tools. There is no mechanism to inject a local tool wrapper into the agent built by build_agent().

Customers should choose based on their use case: option 1 for simple cases where the LLM can reliably pass state, and option 2 when the tool logic is primarily UI-side.

BYOK (Azure Entra ID)

Azure token refresh follows the same 5-line pattern as responses.py:

if (
    model.startswith("azure")
    and AzureEntraIDManager().is_entra_id_configured
    and AzureEntraIDManager().is_token_expired
    and AzureEntraIDManager().refresh_token()
):
    client = await AsyncOgxClientHolder().update_azure_token()

This is applied after model selection and before agent construction.

Shield moderation

Shield moderation is applied via the shield capability integrated into the pydantic-ai agent, rather than a standalone run_shield_moderation_v2() call before inference. The shield capability is attached to the agent during construction and runs as part of the agent’s execution pipeline. If moderation blocks the request, the model response is replaced with the violation message — the client sees a normal text response containing the violation message, not an error event.

Conversation compaction

Compaction is applied only in server mode via apply_compaction_blocking() before inference, the same function used by both the /responses and A2A endpoints. The AG-UI endpoint operates in the same mode as A2A — blocking compaction with no client-side progress event, since AG-UI has its own lifecycle signaling.

In client mode compaction is not applied — the frontend owns the conversation history and is responsible for managing context length.

Telemetry

OTEL spans follow the A2A pattern and are implemented inline in the endpoint handler (not a separate module):

Span Attributes Purpose
agui.handle_request user_id, input, session_id Root span for the request
agui.execute tool_calls_count, tool_calls_names Agent execution
llm.inference model_id, provider_id, input_tokens, output_tokens LLM call

Splunk telemetry is not included in the initial implementation. Only 2 of ~15 endpoints (/responses and /rlsapi) currently emit Splunk events. A Splunk event format module can be added later if the product/analytics team requests it.

Error handling

Scenario AG-UI behavior HTTP status
Auth failure No AG-UI stream; standard HTTP error 401
Authorization denied No AG-UI stream; standard HTTP error 403
Quota exceeded No AG-UI stream; standard HTTP error 429
Shield blocked RunError event in stream with refusal message 200 (stream)
Model not found No AG-UI stream; standard HTTP error 404
OGX unavailable RunError event in stream 200 (stream)
Agent runtime error RunError event in stream 200 (stream)
Context window exceeded RunError event in stream 200 (stream)

Errors that occur before the SSE stream starts (auth, quota, model resolution) return standard HTTP error responses. Errors that occur during streaming are emitted as AG-UI RunError events within the stream — this is consistent with AG-UI protocol semantics where errors during a run are part of the event stream.

Migration / backwards compatibility

Implementation Suggestions

Key files and insertion points

File What to do
pyproject.toml Change pydantic-ai==2.27.1 to pydantic-ai[ag-ui]==2.27.1
src/models/config.py Add AGUI = "agui" to Action enum (next to STREAMING_QUERY)
src/app/endpoints/agui.py New file: AG-UI endpoint handler
src/app/routers.py Import and register agui.router (no version prefix)
src/constants.py Add ENDPOINT_PATH_AGUI constant

Insertion point detail

Action enum (src/models/config.py, next to STREAMING_QUERY):

# Access the streaming query endpoint
STREAMING_QUERY = "streaming_query"

# Access the AG-UI protocol endpoint
AGUI = "agui"

Router registration (src/app/routers.py):

from app.endpoints import (
    agui,
    # ... existing ...
)

def include_routers(app: FastAPI) -> None:
    # ... existing routers ...

    # A2A (Agent-to-Agent) protocol endpoint
    app.include_router(a2a.router)

    # AG-UI (Agent-User Interaction) protocol endpoint
    app.include_router(agui.router)

Dependency change (pyproject.toml):

# Before
"pydantic-ai==2.27.1",

# After
"pydantic-ai[ag-ui]==2.27.1",

Reference patterns

Concern Reference file What to reuse
Protocol adapter endpoint src/app/endpoints/a2a.py Overall structure: auth, agent build, stream conversion, telemetry
LCS middleware integration src/app/endpoints/responses.py BYOK, compaction, model selection, quota
Shield capability src/pydantic_ai_lightspeed/capabilities/ Shield moderation as agent capability
Agent construction src/utils/pydantic_ai_helpers.py build_agent(), captured_output_items()
Response params src/utils/responses.py prepare_responses_params()
Compaction src/utils/conversation_compaction.py apply_compaction_blocking()
Conversation persistence src/utils/conversations.py append_turn_items_to_conversation()
Telemetry format src/observability/formats/responses.py Event builder pattern
OTEL spans src/utils/otel_tracing.py SpanAttributes, SpanEvents, set_span_attributes()

Test patterns

AG-UI-specific test considerations:

Open Questions for Future Work

Changelog

Date Change Reason
2026-10-05 Initial version Widget system spike
2026-10-23 Add client/server conversation modes Client mode aligns with AG-UI protocol (frontend-owned history); server mode enables LCS conversation persistence and compaction
2026-10-23 Shield moderation via agent capability Replace run_shield_moderation_v2() with shield capability integrated into the pydantic-ai agent

Appendix A: AG-UI Event Types Reference

The AG-UI protocol defines ~30 event types organized into categories. The full list is documented at docs.ag-ui.com/concepts/events.

Category Events Purpose
Lifecycle RunStarted, RunFinished, RunError, StepStarted, StepFinished Run lifecycle management
Text TextMessageStart, TextMessageContent, TextMessageEnd, TextMessageChunk Incremental text streaming
Tool calls ToolCallStart, ToolCallArgs, ToolCallEnd, ToolCallResult, ToolCallChunk Tool invocation with streaming args
State StateSnapshot, StateDelta, MessagesSnapshot State synchronization (JSON Patch)
Activity ActivitySnapshot, ActivityDelta Structured activity rendering
Reasoning ReasoningStart, ReasoningMessageStart, ReasoningMessageContent, ReasoningMessageEnd, ReasoningEnd, ReasoningEncryptedValue Chain-of-thought visualization
Sub-agents SubagentStarted, SubagentFinished, SubagentError Nested agent delegation
Special Raw, Custom Passthrough and extension events

Appendix B: AG-UI RunAgentInput Structure

RunAgentInput is the AG-UI protocol’s request format, provided by the ag-ui-protocol package. Key fields as surfaced by AGUIAdapter:

Field Maps to Purpose
threadId AGUIAdapter.conversation_id → LCS conversation_id Conversation identifier
runId AGUIEventStream.run_id Protocol run identifier
messages AGUIAdapter.messages (converted to pydantic-ai ModelMessage) Conversation history
state AGUIAdapter.state Frontend state dictionary
Frontend tools AGUIAdapter.toolset Frontend-defined tools
resume AGUIAdapter.deferred_tool_results Interrupt resume entries (approve/deny/edit)

Appendix C: Comparison with Existing Protocol Endpoints

Aspect /v1/streaming_query A2A (/a2a) AG-UI (/agui)
Protocol Custom LCS SSE A2A (JSON-RPC) AG-UI (SSE)
Request model QueryRequest A2A JSON-RPC RunAgentInput
Event types 8 (token, tool_call, etc.) A2A task events ~30 AG-UI events
Frontend tools No No Yes
State sync No No Yes (snapshot + delta)
Human-in-the-loop No No Yes (interrupt/resume)
Conversation conversation_id param contextId → store threadId → client mode (frontend-owned) or server mode (LCS-persisted)
Target consumer LCS-native clients Other agents Frontend apps
Agent construction build_agent() build_agent() build_agent()
MCP tools resolve_tool_choice() prepare_responses_params() prepare_responses_params()
Compaction apply_compaction() apply_compaction_blocking() apply_compaction_blocking() (server mode only)