[D] Evaluating long-context consistency reasoning in LLMs using full-length narrative evidence

I worked on a long-context reasoning system built and evaluated at IIT Kharagpur as part of the Kharagpur Data Science Hackathon (KDSH 2026).

The goal was to evaluate whether large language models can determine causal and logical consistency between a proposed character backstory and an entire novel (~100k words), rather than relying on local plausibility.

System details:

– Long-document ingestion and evidence aggregation using Pathway

– Local LLM inference using Ollama (Llama 2.5, 7B parameters)

– Fully local, no paid APIs

– Focus on constraint tracking and evidence-based decisions