One MCP Server for Six AWS Inference Backends: Bedrock, SageMaker, GPUs, Inferentia and Trainium

This article introduces a unified Model Context Protocol (MCP) server designed to manage six distinct AWS inference backends, including Amazon Bedrock, SageMaker, and various EC2 instances powered by GPUs, Inferentia2, and Trainium chips. By leveraging a single Python-based MCP server, developers can interact with these diverse environments through a standardized set of eight tools. The server simplifies complex infrastructure tasks—such as listing active resources, checking health status, reading logs, and executing teardown commands—directly from a chat-based interface. By utilizing AWS SSM Run Command, the solution eliminates the need for open inbound ports, enhancing security while maintaining operational control across multiple regions. This approach provides a consistent, agent-driven workflow for managing high-performance model inference, allowing users to monitor billing and performance metrics efficiently without manual terminal intervention. The project is open-source and serves as a practical guide for integrating AI model deployment with modern agentic development workflows.