dirl-diffusion-rl

Enable effective RL for diffusion language models via DiPO (unbiased GRPO for dLLMs) and framework optimizations. FlexAttention accelerates blockwise training, LMDeploy optimizes inference, achieving

SKILL.md

GitHub

Overview

DiRL introduces RL infrastructure tailored for diffusion language models.

Core Technique

DiPO Algorithm: First unbiased Group Relative Policy Optimization for dLLMs.

def dipo_training(model, dataset):
    # Blockwise attention for efficient computation
    # Unbiased logit computation (fixes prior biases)
    # GRPO with dLLM-specific optimizations

Performance

  • State-of-the-art dLLM math performance
  • Outperforms larger autoregressive models

References

  • DiPO: unbiased GRPO for dLLMs
  • Blockwise training optimization