How to Cut AI API Token Costs by 70% Using Prompt Caching & Semantic Routing
Production strategies to slash your monthly OpenAI, Anthropic, and Gemini API bills without sacrificing reasoning performance.
Step 1: Structure System Prompts to Exploit KV Prompt Caching
Modern providers (Anthropic, OpenAI, DeepSeek) offer up to 90% cost reductions on cached input tokens. Place invariant context at the start of your prompt array.
// ✅ OPTIMAL FOR PROMPT CACHING:
// Invariant system prompt + heavy documents placed FIRST in the message array:
const messages = [
{
role: "system",
content: "Large 10,000-token enterprise knowledge base and API specification..."
},
// Dynamic user query placed LAST:
{
role: "user",
content: "Summarize the SLA requirements for tier 1 enterprise customers."
}
];
Step 2: Implement Semantic Routing to Low-Cost Micro-Models
Route routine classification and summarization queries to lightweight models ($0.15/1M tokens) while reserving frontier models ($10.00/1M tokens) for complex reasoning.
function routeQuery(userPrompt: string): 'gpt-4o-mini' | 'gpt-6-astra' {
const isComplexReasoning = /derive|prove|architecture|refactor|benchmark|audit/i.test(userPrompt);
const isLongContext = userPrompt.length > 4000;
if (isComplexReasoning || isLongContext) {
return 'gpt-6-astra'; // Frontier reasoner
}
return 'gpt-4o-mini'; // Fast, 95% cheaper router
}
Step 3: Strip Redundant Whitespace and Markdown Boilerplate
Clean input JSON, remove unnecessary HTML tags, and eliminate decorative formatting before tokenization.
function compactPromptData(data: Record<string, any>): string {
return JSON.stringify(data); // Minified JSON uses ~30% fewer tokens than formatted JSON
}