Frankenmerge is a colloquial and experimental technique in model merging where layers from different pre-trained models are combined, often in a non-standard or 'stitched together' fashion, to create a new model. The goal is to create a larger, more capable model by leveraging the strengths of its constituent parts, even if they have different architectures or were trained for different tasks.
The term 'Frankenmerge' emerged from the open-source AI community, particularly among enthusiasts and researchers experimenting with model merging techniques without the vast computational resources of large labs. The name is a portmanteau of 'Frankenstein' and 'merge,' reflecting the often-unconventional and sometimes-chaotic nature of combining disparate model components.
Frankenmerging is primarily used in the experimental and hobbyist communities. While these models can sometimes achieve surprisingly high scores on leaderboards, they are often less stable and coherent than models trained from scratch or merged using more sophisticated techniques. The practice highlights the community's drive to push the boundaries of model performance through creative and resource-efficient methods.