DARE (Drop And REscale)

What is DARE (Drop And REscale)?

DARE is a merging technique that randomly prunes fine-tuned parameters and rescales the rest, reducing interference when combining many models into a 'Model Soup'.

Where did the term "DARE (Drop And REscale)" come from?

Allows merging many models without degradation.

How is "DARE (Drop And REscale)" used today?

Enables massive multi-skill model combinations.

Related Terms