On: What happens if we remove 50 percent of Llama?

[benob]

You might want to look into "task arithmetic" which aims at combining task-specific models post-training. For example:

https://proceedings.neurips.cc/paper_files/paper/2023/file/d...