Building Production-Ready Open-Weights AI Agents with Gemma, Google ADK, and Cloud Run GPUs

This article provides a comprehensive blueprint for deploying enterprise-grade AI agents using Google's open-weights Gemma models. The author, Robina Mirbahar, details a modular approach to building, connecting, and scaling AI agents on Google Cloud. The guide covers critical production requirements, including grounding agents in live enterprise data via Cloud Firestore, implementing serverless GPU scaling with vLLM on Cloud Run, and enforcing security through Model Armor guardrails. The project, titled 'CloudGuard,' demonstrates how to transition from local prototyping with Ollama to a governed production environment. Key features include automated CI/CD evaluation using the Google Agent Development Kit (ADK), cost-effective QLoRA fine-tuning on Vertex AI, and robust FinOps monitoring for GPU resources. By leveraging these tools, engineering teams can build secure, performant, and cost-efficient AI agents capable of handling complex SecOps and FinOps tasks while maintaining strict data sovereignty and operational control.