Half the Tokens Without Changing the Model: What NVIDIA Did with a Coding Agent's Harness

Researchers from NVIDIA, MIT, and NTU have introduced a new approach to optimizing AI agents specialized in coding. Instead of replacing the model with a smaller one, the team used another AI agent to automatically improve the harness of a coding assistant based on the Pi model. During the experiment, the agent tested 152 optimization strategies for prompts and interactions. Four of these strategies reduced token consumption by nearly half while keeping the original model, leading to a 30% reduction in API costs. Despite a minor 6% drop in performance on the EdgeBench benchmark, the results demonstrate the effectiveness of automated agent infrastructure tuning. The authors note that this method allows for significant resource savings without sacrificing generation quality, though they highlight potential weaknesses in the stability of these solutions when scaled to more complex programming tasks.