LLM Providers' Hidden Session Management Raises Alarms for AI Agent Developers

Armin Ronacher, a prominent member of the Pi team, has published an insightful article exposing a critical, often overlooked issue within the burgeoning field of AI agent development. The article, which resonates deeply with users of agent harnesses like Pi, details how major LLM providers such as OpenAI and Anthropic are increasingly abstracting and centralizing critical aspects of AI session management, leading to significant implications for developers.

The core of the problem lies in several provider-side features. Models like GPT-5.6-SOL now return intermediate thought processes in an encrypted, unreadable format, accessible only to the provider. This not only diminishes transparency into the model’s reasoning but also creates billing opacity, as users pay for tokens they cannot audit. Furthermore, optional server-side features, including managed web search tools, context compaction, and conversation state APIs (where store: true is often the default), consolidate session data on the providers’ servers. While offering convenience by reducing data transfer, these practices severely compromise session ownership, portability across different models or providers, and the ability to audit historical interactions. Ronacher suggests this trend is partly motivated by providers’ efforts to prevent model distillation, ultimately leading to reduced user control and potential vendor lock-in for critical AI workflows.