Migrating to v0.8.0
Table of Contents
Context windows for conversation compaction
In v0.8.0, the context window that conversation compaction uses for a model
is set as metadata.context_window on the model’s
inference.providers[].allowed_models entry. The top-level
inference.context_windows map is deprecated. It is still accepted, applies
to a model whose entry sets no window, and logs a startup warning.
| v0.7.x | v0.8.0 |
|---|---|
inference.context_windows["<provider id>/<model name>"] |
metadata.context_window on the allowed_models entry named <model name> of the inference.providers entry with id <provider id> |
| A model missing from the map never compacts | The window is looked up in the model data bundled with the service, otherwise 128,000 tokens is assumed |
Move a window only for a provider that lightspeed-stack.yaml already
declares under inference.providers, and keep the other windows in
inference.context_windows for now. An entry added only to carry a window
changes the OGX configuration:
- Adding
inference.providersto a file with no other synthesis input (vector_store.providers, anogx.configblock) makes it a unified synthesis input. Where the OGX server’s run configuration is generated from the same file (the OGX image entrypoint,docker-compose.yaml, the Prow manifests), the entrypoint then synthesizes the run configuration from the default baseline instead of enriching the mountedrun.yaml. - Wherever the OGX configuration is synthesized from the file (unified
library mode, and the entrypoint above), the entry replaces the provider of
the same id that comes from
ogx.config.profileor the baseline. Only itsapi_key_env,extraandallowed_modelsreach OGX; a typedconfig:block is not emitted yet, so settings of the replaced provider such asbase_urlorapi_versionare lost. Itsallowed_modelsalso becomes the provider’s allow-list, which hides every model it does not list.
Before (v0.7.x):
inference:
context_windows:
rhaiis/granite-3.3-8b-instruct: 131072
After (v0.8.0), in the allowed_models of the provider’s existing entry:
inference:
providers:
- type: vllm
id: rhaiis
# ... the entry's other settings, unchanged ...
allowed_models:
- provider_model_id: granite-3.3-8b-instruct
metadata:
context_window: 131072
A key of the map splits at its first /: the part before is the provider
id (which defaults to type), the rest is the model name (which defaults
to provider_model_id). A legacy two-file configuration
(ogx.library_client_config_path) cannot declare inference.providers;
migrate it to the unified lightspeed-stack.yaml first.
With compaction enabled, a model that has no configured window now compacts
at threshold_ratio of the window found for it instead of never. Set
metadata.context_window for self-hosted models whose window is smaller than
128,000 tokens. See the
conversation compaction guide
for the full resolution order.
GET /a2a removed
In v0.8.0, the GET /a2a route is gone. It never ran an A2A operation: the
request was forwarded to the A2A SDK application, whose JSON-RPC route accepts
POST only, and came back as 405 Method Not Allowed. No working call is lost.
| v0.7.x | v0.8.0 |
|---|---|
GET /a2a with accepted credentials: 405, text/plain, Allow: POST |
405, application/json ({"detail":"Method Not Allowed"}), Allow: POST |
GET /a2a without accepted credentials or the a2a_jsonrpc action: 401 or 403 |
405 as above; the router answers before authentication runs |
GET /a2a with no agent card configured: 500 |
405 as above |
operationId handle_a2a_jsonrpc_a2a_get in the OpenAPI document |
Removed |
Use POST /a2a for JSON-RPC calls, and GET /.well-known/agent.json or
GET /.well-known/agent-card.json to fetch the agent card.
/v1/info API
GET /v1/info now describes the Lightspeed Core Stack service only and no
longer contacts OGX. The ogx_version field is removed with no replacement:
| v0.7.x | v0.8.0 |
|---|---|
name, service_version, ogx_version |
name, service_version |
503 Unable to connect to OGX while OGX is unreachable |
200 with the service name and version |
Update clients that read ogx_version, and do not use /v1/info to detect
whether OGX is available: GET /readiness reports that.