Relevance Gone After Adding Files to Package

(imported topic written by MBARTOSH)

Using the software distribution dashboard, I deleted all of the files for one of my packages, and then added the files back using manage tasks. In the process, all of my custom relevance for the task was deleted. How can I prevent this from happening? I now have to recreate the relevance.