RETRO Enables Smaller LLMs Matching GPT-3

DeepMind's RETRO research, described in this article on April 5, 2026, shows retrieval-augmented language models can match GPT-3 performance while using far fewer parameters. RETRO uses a 2-trillion-token key-value database and BERT-based sentence embeddings to retrieve neighbor chunks that augment a 7.5B decoder, achieving parity with a 185B GPT-3 Da Vinci. This approach reduces training cost and enables smaller, deployable LLMs.