← All news·2026-11-22·3 min read

Nvidia Researchers Say Giant Context Windows Are Often Unnecessary

According to The Decoder, a team at Nvidia says large context windows are slow and expensive. In their tests, periodic summaries and targeted retrieval did as well.

airesearchllm

Researchers at Nvidia, the company best known for the chips that run AI, say the huge memory of today's language models is often more than needed. That memory is called the context window. It is the amount of text a model keeps in view while it writes a reply. The Decoder, a German news site about AI, reported the claim.

These windows have grown fast. GPT-4 from OpenAI launched in March 2023 with 8,192 tokens, the small pieces of text that models count. GPT 5.6 Sol now handles about one million. The article says that much text makes a model slow and expensive, and leads to mistakes. In many tests, regular summaries and targeted lookups worked about as well as the full context. If that holds, a model's skill at picking what to read may matter more than the size of its window.

The authors propose a kind of model that works from short notes instead of a long context. The full article is behind a paywall. So the researchers' names, the title of their paper and the costs in their tests are unknown. For now this is the team's own claim, and no one else has tested it yet.

Source: the-decoder.com

Free course

Stop reading about AI — start building with it

The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.

Start free →
EAEvgenii Arsentev

Author

Evgenii Arsentev

PhD · Founder & CEO, Dad4Kids