GitHub Improves Copilot Context Handling and Model Routing

GitHub published a blog post on June 17, 2026, authored by Ryan Caldwell and Bhavya U, detailing harness-level token-efficiency improvements in Copilot for VS Code. Per the post, each new model generation consumes more tokens per task than the last, making harness-level gains increasingly necessary to offset rising costs and latency. The post covers prompt caching, deferred tool loading (tool search), and WebSocket transport for OpenAI models, plus analogous improvements for Anthropic-backed models.

In agentic sessions, system instructions, tool definitions, repository context, and conversation history repeat across turns as a prompt prefix. GitHub now retains cached OpenAI model state for up to 24 hours using prompt_cache_retention: "24h", compared to a default 5-10 minute window. This keeps the cache warm after pauses and reduces the fraction of each request billed at the full (uncached) rate -- which is up to 10 times more expensive than the cached rate for supported models. The gains are largest after long pauses: the post reports relative cache-hit-rate increases of +679% at 40-60 minutes for GPT-5.4.

For Anthropic models, which require explicit cache_control breakpoints rather than automatic prefix detection, GitHub reworked placement to anchor at the four most stable boundaries: end of tool definitions, end of system prompt, and two rolling anchors on the two most recent cacheable messages. This brought cache hit rates in agentic workloads to around 94%, per the post.

Agents can load 100+ tools; sending every tool's full JSON schema on every turn consumes significant context. Tool search defers full schemas until the model issues a tool_search call, loading only the matched tools. For OpenAI models (GPT-5.4 and newer), a four-day experiment found roughly 9-10% per-turn token reduction and ~5% time-to-complete improvement. For Anthropic models, GitHub moved the search client-side, backed by an internal embedding model for semantic matching. A seven-day experiment found ~11% per-turn and ~18% per-user total token reduction; a subsequent two-week rollout found additional 1-4% latency reductions for Claude Opus 4.6 and Sonnet 4.6, and a 4% reduction in user error rate for Sonnet.

GitHub added persistent WebSocket connections for GPT-5.2 and newer, replacing repeated HTTP round trips across sequential agent steps. During initial rollout, time to first token fell by 16-19% at P50 for GPT-5.3-Codex and GPT-5.4. Active-user and two-day engagement metrics showed statistically significant increases.

Per the post, GitHub plans to route "whole classes of work" off the main agent to specialist subagents for narrow tasks -- workspace search, command execution, result summarization -- running on smaller, cheaper models. The team also plans transparency features that surface cache-state and per-action cost to help developers avoid inadvertent cache cold starts.