Microsoft Shows Model Upgrades Can Raise Agent Costs

Model routing is becoming an operations problem, not just a benchmark decision. Microsofts fresh Copilot Chat experiment gives AI teams a concrete example of why a lower per-token price can still produce a higher bill when an agent plans longer, loops more, or emits more intermediate work. For LDS readers building agents, the useful signal is the shape of the variance: the risky part is not only average cost, but the outlier run that can turn one automated task into a budget surprise.

In a July 6 developer post, Microsoft said it ran 150 agent tasks across 15 scenarios using GitHub Copilot Chat in VS Code on Windows. The tasks covered architecture and design work grounded in Microsoft Learn documentation, plus SharePoint Framework project upgrades. Microsoft compared Claude Sonnet 4.6 and Claude Sonnet 5, then evaluated runs with binary gates and quality dimensions while pricing each run from actual per-turn token data.

The results were mixed rather than linear. On architecture tasks, both models reached 75% task completion, while output quality was 90% for Sonnet 4.6 and 78% for Sonnet 5. On code upgrades, Sonnet 5 reached 100% completion versus 60% for Sonnet 4.6, but its average cost per run was $2.01 versus $0.55. Microsoft also reported large token-consumption swings, including a same-scenario Sonnet 5 run that consumed 6.6 million tokens while another consumed 16,000.

The post is a strong argument for treating model upgrades like dependency upgrades. Teams should replay representative workloads, inspect completion and quality separately, and measure token distribution before changing defaults. A model can be the right choice for high-precision upgrade work and the wrong default for routine architecture review. The procurement and platform question is therefore not which model is newest, but which model has the best cost, variance, and quality profile for each agent workload.