From Attention to Compression: Learning the Hidden Runtime of AI

Описание: What if a transformer could grow new kinds of attention patterns without adding layers, pretraining were really a form of compression, and a model could begin to behave like a computer with its own latent runtime? In today’s episode, we connect three striking papers that push AI in very different directions: more expressive attention, a new information-theoretic view of learning, and the first steps toward neural computers. Together, they hint at a future where models think, remember, and act in far richer ways.