• Authors propose a new preprint exploring math behind large language models. • Question: how to model transition from probability distributions on text to syntax and semantics. • Language viewed as algebraic/compositional, with concatenation acting like multiplication. • The paper aims to bridge statistics, category theory, and linguistic structure. • Upcoming posts will tour the formal ideas presented in the preprint. • The work builds on recent successes of state‑of‑the‑art LLMs.
Article Summaries:
- A new preprint by John Terilla, Yiannis Vlassopoulos, and the author proposes a mathematical framework to explain how large language models translate probability distributions over text into syntactic and semantic structure. The paper frames language as an algebraic, compositional system, drawing on ideas from abstract algebra and category theory. It revisits John Firth’s observation that a word’s meaning is revealed by its “company” and models this through algebraic constructs such as two‑sided principal ideals. The authors outline how these concepts can be used to formalize the relationship between statistical language models and linguistic information.
Sources: