Medusa: A Framework for Collaborative Development of Foundation Models with Automated Parameter Ownership Assignment

Dezhi Ran, Yuan Cao, Yuzhe Guo, Yuetong Li, Mengzhou Wu, Simin Chen, Wei Yang, Tao Xie

Summary

Medusa supports collaborative foundation-model development by tracking fine-tuning branches and assigning ownership of model parameters. Merging-aware masks guide training so branches modify designated parameters. Evaluation across five datasets and three models shows improvements over post-training model merging, including a 3.19-point absolute improvement in merged-model performance.