Building Sarrera: Self-Hosted Enterprise AI Inference Gateway with RBAC, Token Quotas & Telemetry

Sarrera is a new open-source, self-hosted enterprise AI inference gateway designed to help engineering teams manage private AI deployments. Addressing common challenges like data privacy, uncontrolled cloud billing, and hardware fragmentation, Sarrera provides a unified platform for managing local AI infrastructure. Built on a Docker Compose architecture, it integrates LiteLLM for routing, Caddy for secure entry, Langfuse for observability, and Open WebUI for interaction. The platform enables organizations to enforce multi-tier subscription quotas, implement role-based access control (RBAC), and perform intelligent load balancing across heterogeneous GPU and CPU clusters. By decoupling client IDEs from physical hardware, Sarrera allows teams to maintain control over token usage and costs while ensuring compliance. The project is available on GitHub and offers a streamlined deployment process for teams looking to build a private, scalable, and observable AI hub in under five minutes.