Collection date metadata strongly affects temporal question answers

Collection-Date Assignment Redirects Answers in Temporal QA

Computation and Language

Summary

When computers answer questions about when facts happened, they often look at the date when they got the information, not when the fact itself changed. The authors show that changing the assigned collection date (the date the information was taken) can make language models give very different answers, even if the facts haven’t changed at all. This means these models might rely too much on metadata dates rather than actual content changes. The authors tested several large language models and found consistent effects, offering a method to check how dates influence answers about time.

What this means in practice

  • For software engineers: Detect and correct metadata-related errors in temporal question answering systems for more accurate date-sensitive responses.
  • For content management teams: Audit and improve how document acquisition dates are used in systems that answer time-based questions to avoid misleading answers.

Authors

Minsu Oh, Seunghui Jwa, Soo Kyun Kim, Yeo-Chan Yoon

Abstract

A collection date records when a source revision was acquired, not when its contents changed. Can assigning this metadata redirect temporal question answering despite unchanged evidence? Swapping the question day and following day between two revisions lowers later-reference exact match (EM) by 37.6-51.4 percentage points across five open-weight LLMs on 595 Wikipedia pages, replicated on 128 disjoint pages. Previous-day swaps retain 29.0-39.6-point losses on 497 pages. Seven prompt variants, including explicit ignore-collection instructions and archived-revision provenance, retain the effect. Fictional answers and English month-name question dates show that remembered answers and identical date strings are unnecessary. Matched thinking reduces Qwen3.5's penalty from 30.1 to 11.1 points, while native reasoners vary. Two API LLMs show small Wikipedia swap effects and rarely shift to earlier answers on fictional records. Large swap-induced losses concentrate around the question date; swaps between distant dates have much smaller effects. Adding fields containing one observed acquisition date lowers EM relative to omission in three of five LLMs after correction. The diagnostic isolates metadata effects from changing evidence or missing the relevant revision, and provides a practical audit for temporal QA inputs.