Lecture: #9 Language Models for Code Generation - ScaDS.AI Dresden/Leipzig

Описание: In this lecture, Klaudia-Doris Thellmann and Bernhard Stadler teach you about Language Models for Code Generation.

Efforts to automatically generate source code from natural language instructions to overcome the human-machine language barrier have existed since the early days of computer science. As a result, numerous approaches have emerged over the last decades, ranging from statistical methods with focus on rule induction or probabilistic grammars to artificial neural networks. In the last few years, large-scale pre-trained neural language models have shown impressive performance on a variety of natural language processing tasks and have also become the basis for a growing number of programming-related tasks such as code completion or synthesizing code from natural language descriptions.

Klaudia-Doris Thellmann and Bernhard Stadler give an overview of state-of-the-art language models for program synthesis. Furthermore, they present basic characteristics of these models and discuss several of their limitations. A critical weakness of these language models is systematic reasoning, which is crucial for understanding the programming task at hand and generating program code. One possible direction of research that could help alleviate these limitations is the inclusion of structural knowledge – an attempt they make in this regard and briefly introduce.