RLHF

Mar 30, 2026 LLM Engineering 48 min read

LLM Engineering (4): Post-training — SFT, DPO, RLHF, RLAIF

What SFT, DPO, RLHF, and RLAIF each actually optimize, when reward models fail, KL constraints, the LoRA-vs-full-FT debate, and the production post-training recipes that ship in 2026.

Sep 25, 2025 Reinforcement Learning 40 min read

Reinforcement Learning (12): RLHF and LLM Applications

How RLHF turned base language models into ChatGPT and Claude: the SFT→Reward-Model→PPO pipeline, the Bradley-Terry preference model, the DPO closed-form derivation, RLAIF and Constitutional AI, reward hacking and …

Aug 26, 2025 Reinforcement Learning 32 min read

Reinforcement Learning (6): PPO and TRPO — Trust Region Policy Optimization

Why PPO became the most widely used RL algorithm -- from TRPO's theoretical foundations through natural gradients to PPO's elegant clipping mechanism, plus its role in RLHF for large language models.