DoRA decomposes the pre-trained weight into magnitude and direction components, applying LoRA only to the directional component. This often yields better performance and stability than standard LoRA.
A 2024 advancement in parameter-efficient fine-tuning.
Gaining traction for tasks requiring complex reasoning where standard LoRA might fall short.