Gemma 4 Inference on AWS: Bedrock, SageMaker, GPUs, Inferentia and Trainium Behind One Strands Agent

This article presents a comprehensive performance survey of running Gemma 4 LLM inference across six distinct AWS environments. The author utilizes a unified 'Strands' agent to benchmark Amazon Bedrock, SageMaker real-time endpoints, vLLM on EC2 GPU instances (g6 and g5g), and custom-ported models on AWS Inferentia2 and Trainium chips. By standardizing prompts and measurement scripts, the study highlights the practical differences in setup, latency, and throughput across these diverse backends. Key findings emphasize that while managed services like Bedrock offer ease of use, self-hosted solutions on specialized hardware like Trainium provide competitive performance if configured correctly. The author also provides technical insights into managing AWS capacity, handling port forwarding via SSM, and overcoming specific challenges when deploying models on Neuron-based accelerators. This guide serves as a practical roadmap for developers looking to optimize LLM deployment strategies on AWS infrastructure.