OMP Telemetry & Token Usage Extraction

When building OMP extensions (like langfuse.ts) or telemetry tools that tap into the after_provider_response hook, you MUST handle diverse LLM provider payloads to prevent token loss.

The Problem

Providers (OpenAI, Anthropic, Gemini) and routers (Vercel AI SDK) format token metrics differently, especially during streaming. If you only check event.usage, you will lose data.

The Solution

Always use a deep-coalescing approach to extract usage:

// 🧠 Meticulously extract usage from ALL possible payload variations
const rawUsage = event.usage ?? (event as any).metrics?.usage ?? (event as any).metrics ?? (event as any).response?.usage;
const actualUsage = rawUsage && typeof rawUsage === "object" && 'usage' in rawUsage ? (rawUsage as any).usage : rawUsage;
const usage = actualUsage as Record<string, any> | undefined;

// Safely extract deeply nested tokens (e.g. OpenAI caching)
const cacheDetails = usage?.prompt_tokens_details;
const cacheRead = Number(usage?.cacheRead ?? usage?.cache_read ?? usage?.cachedTokens ?? cacheDetails?.cached_tokens ?? 0) || 0;
const cacheWrite = Number(usage?.cacheWrite ?? usage?.cache_write ?? 0) || 0;
const inputTokens = Number(usage?.input ?? usage?.inputTokens ?? usage?.promptTokens ?? usage?.prompt_tokens ?? usage?.promptTokenCount ?? 0) || 0;
const outputTokens = Number(usage?.output ?? usage?.outputTokens ?? usage?.completionTokens ?? usage?.completion_tokens ?? usage?.candidatesTokenCount ?? 0) || 0;

Key Takeaway: Never assume token counts are perfectly uniform; always fallback through metrics, response.usage, and nested prompt_tokens_details.