Fitting the Documents In Doesn't Mean Using Them All

Find the passage that supports this answer, then read it in context.

A larger context window sounds useful when you're working through a pile of documents. There's room for more of the material, so you don't have to leave as much out. But room in the input doesn't tell you how reliably the model will use a particular passage when answering your question.

Input capacity and the ability to use information for a task are different things. A controlled experiment makes that distinction easier to see than a window-size specification does.

Move the passage, keep the answer

Nelson F. Liu and colleagues' Lost in the Middle: How Language Models Use Long Contexts was published in TACL in 2024. The experimental details discussed here come from the authors' arXiv v3, dated 20 November 2023.

In their multi-document question-answering task, the researchers supplied a question and 10, 20 or 30 short Wikipedia passages. The setup designated one passage as containing the annotated answer, with the others serving as distractors. They changed where that answer-containing passage appeared in the input while keeping the target answer unchanged.

Moving the passage didn't make the question require a different answer. Yet the tested models often performed better when the relevant material was at the beginning or end, and worse when it was in the middle. Extended-window variants weren't necessarily better at using material that also fitted within their smaller-window counterparts.

The scoring asked whether an annotated correct answer appeared in the model's output. That is useful for this retrieval-and-answering question; it doesn't assess whether the output gives a complete, well-supported argument.

The point is more specific than “long documents are difficult.” The needed passage was already present in the controlled text input. Changing its location could still change whether the model produced the expected answer.

Repeating the question helped a different task

The researchers also tested lookup in a collection of random identifier pairs. Both keys and values were UUID strings; given one key, the model had to return its paired value. This stripped away much of the meaning involved in answering a question from prose. Some tested models already performed nearly perfectly on this task.

Placing the query both before and after the data brought lookup performance close to perfect across the tested models and collection sizes. The same arrangement did little to change the position-related performance pattern in multi-document question answering.

So successfully retrieving an identifier doesn't establish that a model will reliably use a passage to answer a natural-language question. And a prompt arrangement that helps one task isn't a general guarantee. The findings don't supply a reliable recipe of putting everything important at an edge, repeating the question, or processing every task in batches.

These are results from older models and particular tasks, not a verdict on every tool available in 2026, summarization or programming. The experiments controlled actual text inputs; they didn't test whether a service had included an uploaded file or followed a supplied link. That is a separate question about what material the service made available.

Make one answer easier to examine

For everyday document work, an editorial suggestion is to begin with one concrete question whose supporting passage you can read back. If you're reviewing interview notes, for example, you might ask:

Do any interviewees describe a problem after their shift schedule changed? Quote the passage supporting your answer and identify the document and its location. Include relevant qualifications or contrary accounts; don't infer an experience that isn't recorded.

Then return to the original passage, with its surrounding context, and see whether it supports the claim. A document name or locator is useful only if it takes you back to the words being used as evidence.

This lets you examine the basis of that answer. It doesn't show that every document was sufficiently considered. If your real question requires synthesis across the interviews, keep the relevant qualifications and contrary evidence in view as well. Shortening the material by removing those passages could change the substance of the question, even if the resulting input is easier to fit.

This approach is our editing advice, not a universal intervention validated by the paper. The aim is a specific answer with evidence you can inspect, while leaving broader coverage as something still to establish.