Prism automatically caches eligible prompt prefixes. Repeated system prompts,
tool definitions, repository context, and conversation history can reuse work
when their leading tokens stay identical.
Keep the prefix stable
Put stable content first and frequently changing content last:
- System instructions
- Tool definitions
- Repository or reference context
- Conversation history
- The newest user message
Changing one token near the beginning of the prompt prevents reuse for the
content that follows it.
Set a conversation key
For Chat Completions, send the same prompt_cache_key on every turn in one
conversation:
Use an opaque, stable identifier. Do not put email addresses, API keys, or
other secrets in the cache key. prompt_cache_key accepts up to 512 bytes.
cache_ttl accepts 5m, 30m, 1h, 6h, or 24h. Invalid values return
400.
You can also send the identifier as a header:
When both are present, x-session-id takes precedence over
prompt_cache_key. Use one mechanism consistently for a conversation. A cache
key provides affinity and is not an idempotency key.
- Serialize tool schemas deterministically.
- Keep tool order stable between turns.
- Avoid timestamps and request IDs in system prompts.
- Append new conversation turns instead of rebuilding older messages.
- Reuse the same model within a conversation.
- Keep large, shared context before the newest user input.
Multi-tenant safety
Generate a different key for each tenant and conversation. Never reuse one
customer’s key for another customer, even when their prompts look similar.
The cache is an inference optimization. Your application remains responsible
for conversation storage, retention, and access control. Last modified on September 9, 2026
