Open-dLLM is Fred Zhangzhi Peng's Apache-2.0 release of what it calls the most open diffusion-based language model to date: the entire pipeline from raw data to pretraining code to inference, evaluation, and checkpoints lives in a single repository. With 620+ GitHub stars and an active 0.5B-parameter code model called Open-dCoder, it is the first practical, reproducible reference point for non-autoregressive code generation that anyone outside a frontier lab can actually retrain. ## Why Diffusion LMs Matter for Code Most current code LLMs are autoregressive: they generate one token at a time, left to right. Diffusion language models flip this by treating generation as iterative denoising over a fixed-length sequence of mostly-masked tokens, which means they can in principle revise earlier tokens after seeing later context. For code generation, where infilling and structural rewrites are common, this is a natural fit. Open-dLLM is the first serious attempt to make that approach reproducible end-to-end at small scale. ## What Open-dCoder Actually Is Open-dCoder is a 0.5B-parameter masked diffusion code model initialized from Qwen2.5-Coder and adapted to a Masked Diffusion Model (MDM) objective with uniform masking ratios. It is trained on the FineCode dataset, a curated high-quality code corpus the project ships alongside the model. Reported numbers include 20.8% HumanEval Pass@1 and 32.5% on HumanEval Infill, which puts a 0.5B diffusion model in the same range as autoregressive baselines several times its size on infilling tasks specifically. ## The Full Stack, Open The defining feature of Open-dLLM is not the model size or even the benchmark numbers; it is that every stage is open. The repository ships the data preprocessing pipeline, pretraining code, fine-tuning recipes, inference scripts, an evaluation harness covering HumanEval, MBPP, and HumanEval Infill, and the actual checkpoints. Researchers can take a clone of the repo and rerun the entire pipeline from raw data to evaluation, which is rare for diffusion LM work and effectively absent for diffusion code LMs. ## Representation Alignment for 4x Speedup The project includes a technique called representation alignment that lets autoregressive language models be adapted into diffusion LMs with about a 4x training speedup compared to training a diffusion LM from scratch. This is the trick that makes initializing from Qwen2.5-Coder work, and it is one of the more practical contributions in the codebase because it lowers the cost of experimenting with diffusion LM architectures by a meaningful factor. ## Evaluation Suite The evaluation harness covers standard code completion (HumanEval, MBPP) and code infilling (HumanEval Infill), which is important because infilling is where diffusion LMs structurally have an advantage. Having a unified eval suite that works for both autoregressive and diffusion code LMs makes it possible to do apples-to-apples comparisons, which the field has lacked. ## Where It Fits Open-dLLM is the right starting point for anyone who wants to do research on diffusion language models for code without building the whole stack from scratch. It is also useful as a teaching reference because the masked diffusion training loop, the alignment technique, and the evaluation harness are all in one place and small enough to read end-to-end. At 0.5B parameters it is not a production code assistant, but it is a complete experimental platform. ## Limitations Open-dCoder at 0.5B parameters does not match large autoregressive coders on standard completion benchmarks: 20.8% HumanEval Pass@1 is below what current frontier models reach. The training corpus and recipes are sized for accessible reproduction rather than for chasing leaderboard numbers, so anyone hoping to drop Open-dCoder into a production coding workflow will find it underpowered. Diffusion LM inference is still slower per generated token than autoregressive generation in many setups, and the representation alignment speedup applies to training, not inference. Apache-2.0 license keeps it permissive, but the practical use case is research and reference, not deployment.