Different Weight Geometries in LLM Training Methods

Different Weight Geometries with Similar LLM Training Methods

Research by AlexWortega shows that DPO, GRPO, and DAPO optimization methods create a different model weight structure compared to SFT or RFT, even when using the same data.

Compiled by Sergey KostenchukPublished 2026-06-21Updated 2026-06-22

2026-06-21 Research HuggingFace

🧠 Different Weight Geometries with Similar LLM Training Methods

Research by AlexWortega has shown that DPO, GRPO, and DAPO methods form a fundamentally different model weight structure compared to SFT, RFT, and DFT, even on identical data. This effect is stable and independent of hyperparameters.

🌍 This is critically important for developing transfer learning methods and assessing model reliability, as different training methods create different internal representations of knowledge.

👤 This explains why models with the same accuracy can behave differently in new tasks — their "internal map" of knowledge is constructed differently.

Source 1: https://huggingface.co/spaces/AlexWortega/same-data-different-losses Source 2: https://x.com/justALEXWORTEGA/status/2068790635570561429

Sources