Accuracy often varies by position: beginning, middle, and end of the haystack. Passing NIAH does not prove reasoning over the whole document — only that a planted fact can be found. Builders still use it as a cheap long-context smoke test.
A test that hides a fact in a long context and asks the model to retrieve it, measuring whether the context window is actually usable.
Accuracy often varies by position: beginning, middle, and end of the haystack. Passing NIAH does not prove reasoning over the whole document — only that a planted fact can be found. Builders still use it as a cheap long-context smoke test.