Hierarchical Curriculum Learning for Multi-Document Reasoning in Large Language Models

This paper addresses the challenges of evidence dispersion, attention shift, and inference inconsistency in long text and multi-document reasoning scenarios. It investigates the problems that large language models often encounter under conditions of extremely long contexts, multi-source information conflict, and structural complexity, and proposes a hierarchical, curriculum-based fine-tuning algorithm framework. This framework organizes the input into a hierarchical structure of questions and multi-document contexts. At the representation level, it constructs a three-level convergence path of tokens, fragments, and documents to form structured memory and mitigate semantic drift caused by context expansion. At the reasoning level, it introduces a question-guided evidence scoring and weight aggregation mechanism to achieve differentiable selection across document fragments and global evidence vector construction, thereby strengthening the alignment of key evidence and suppressing redundant interference. At the training organization level, it employs a curriculum strategy, progressively scheduling samples according to difficulty levels, enabling the model’s capabilities to gradually transition from local consistency to cross-document evidence integration and overall induction. Comparative experimental results show that this method exhibits more stable performance in multi-document evidence localization and inference consistency evaluation, validating the effectiveness of hierarchical modeling and curriculum scheduling in shaping multi-document reasoning capabilities.