- Science & Technology
Current chain-of-thought models often sound persuasive without actually reflecting the reasoning behind the answer, so this work aims to make the reasoning itself carry the information the model needs to be right.
In this work, we study how to make chain-of-thought reasoning more informative by introducing a Markovian language model framework with a reasoning bottleneck: the model must answer from the chain of thought alone, making the reasoning more causally load-bearing rather than optional. We find that this approach recovers much of the performance of stronger baselines while producing CoTs that are more sensitive to perturbation and transfer better across models.
Check it out here:
Title: “Markovian Transformers for Informative Language Modeling”
Paper: https://lnkd.in/gY6W-nbR
Authors: Scott Viteri, Max Lamparth, Ph.D., Peter Chatain, and Clark Barrett.
Cite the Paper [APA]:
Viteri, S. W., Lamparth, M., Chatain, P., & Barrett, C. (2026). Markovian transformers for informative language modeling. In Proceedings of the Fourteenth International Conference on Learning Representations (ICLR 2026).
