Model Plane Takes Flight: A Unified Control Plane for Fleet-Scale AI Inference

A new open-source project, Model Plane, has been unveiled, addressing the significant challenges of serving large-scale AI models across distributed and scarce GPU infrastructure. Traditional methods for deploying single models on single clusters are well-understood, but scaling these deployments to a ‘fleet’ of GPU clusters — scattered across regions, clouds, and on-prem hardware — becomes a complex, manual, and often prohibitive task. Model Plane is designed to streamline this process by establishing a clear separation of concerns: a ‘platform side’ for fleet operators managing hardware and cluster provisioning, and a ‘developer side’ for engineers and data scientists focused solely on model deployment. The project aims to abstract the underlying infrastructure complexities, allowing developers to declaratively request GPU resources without needing to understand specific cluster or instance types, while still giving platform teams granular control over their compute resources.

At its core, Model Plane operates via a central control plane that manages a fleet of ‘Inference Clusters.’ Hardware capabilities are defined through reusable ‘Inference Classes,’ akin to Kubernetes StorageClasses, which describe GPU types and provisioning methods (e.g., Nvidia L4, L40TS on EKS, GKE, or existing clusters). A sophisticated scheduler then intelligently matches developer-declared model requirements (e.g., GPU memory) against these platform-defined hardware classes and cluster availability, even leveraging advanced features like AWS capacity block reservations and EFA for high-speed networking. Each managed cluster receives a complete, OpenAI-compatible serving stack, eliminating manual assembly. Developers deploy models using standard Kubernetes pod templates, which Model Plane wraps with routing and then exposes through a single, unified endpoint on the control plane, simplifying access to geographically dispersed replicas. Although currently in V1 Alpha 1, Model Plane demonstrates a compelling approach to managing fleet-scale inference, inviting community contributions and feedback to evolve its potential.