Public preprint and research harness

Context-Window Research

Question: can less-relevant context help a model distinguish what matters, even when the number of items stays fixed?

Read the public preprint, version 1 · Inspect the harness · Archived paper

What was compared

The published study reports 2,420 confirmatory trials across 11 model configurations using 661 anonymized workplace items scored for relevance. The main comparison holds the prompt at ten items and changes its composition: high-relevance items only versus a mixture of high- and same-domain low-relevance items.

This is a test of composition, not a demonstration that adding more tokens always improves an answer.

A reported finding, with its denominator

In the paper's all-11 paired comparison, the 50:50 condition improved relevance-score accuracy by 0.077 relative to the all-high-relevance condition (n = 220 paired comparisons). Those cells are not independent. The paper also reports a nine-model-family aggregation and explicitly treats it as a within-corpus descriptive comparison, not population inference.

The 220 paired comparisons and 2,420 study trials are different denominators. Neither is a count of customers or independently replicated studies.

The product implication

Evaluate what belongs in a context window against the actual task. A relevance-only filtering rule can remove information that helps discriminate between items; a larger window is not automatically a better one.

The harness makes composition, model calls, and extraction scoring inspectable. It is an artifact to examine and adapt, not a promise that its arrangement strategy generalizes to another domain.

What remains limited

  • One corpus and task family; this does not establish a universal context-design rule.
  • A public preprint is not peer review or independent validation.
  • Later local assurance, rater-reliability, and calibration work is separate from the published version. Its completion is not claimed here.
  • This site does not provide private corpus items, internal mappings, or unreleased result packages.
  • The paper distinguishes measured timing runs, an operational baseline, and estimated token reductions. This page does not turn those into a verified business-impact claim.

Source: Dvir Shamay, Token Optimization and Context Window Management in Multi-Agent AI Workflows, arXiv:2608.17188v1, submitted August 17, 2026.

context-economicsllm-evaluationai-systems

Start a conversation about this work.