[LCTES'26] Scheduled Partial-Credit RL for Reliable Code Generation with Small Language Models (WIP)

Описание: Scheduled Partial-Credit RL for Reliable Code Generation with Small Language Models (WIP) (Video, LCTES 2026)
Suryansh Singh Sijwali, Suman Saha
(Pennsylvania State University, USA; Pennsylvania State University, USA)

Abstract: Small language models (SLMs, ≤1.5B parameters) are attractive for embedded and resource-limited development workflows because they can run under single-GPU or CPU budgets and be adapted without distributed training. SLM-based code generation is brittle under strict sandboxed evaluation, and reinforcement learning (RL) with binary test rewards is often too sparse to train SLMs reliably. This WIP paper presents a reliability-first RL framework for SLM code generation built around a joint reward. The functional term assigns intermediate credit to near-miss outcomes for syntax validity, crash-free execution, and output production, while a static-analysis term discourages unsafe shortcuts during training. On DeepSeek-Coder-1.3B evaluated on 100 stdin-style APPS+ prompts, a binary-to-partial-credit curriculum improves syntax validity to 63% and produces solutions that pass at least one test in 9% of prompts in a single generated attempt. In contrast, binary-reward PPO regresses below a supervised fine-tuning baseline and partial-credit training from scratch reaches only 27% syntax validity.

Article: https://doi.org/10.1145/3814943.3816167

ORCID: https://orcid.org/0009-0005-7739-1657, https://orcid.org/0009-0005-9440-6785

Video Tags: Partial-Credit Reward Shaping, Reward Scheduling, Curriculum Learning, Reinforcement Learning (PPO), Small Language Models, Code Generation, doi:10.1145/3814943.3816167, orcid:0009-0005-7739-1657, orcid:0009-0005-9440-6785

Presentation at the LCTES 2026 conference, June 15-16, 2026, https://pldi26.sigplan.org/home/LCTES...
Sponsored by ACM SIGPLAN.