To be honest, I think when adding NEW content to the game we must stop modifying the existing game files, if possible. Instead, we should start distributing new files and only if really necessary distribute original (modified) game files. This is a legacy from the days without S3D (self pimp, I know) in my opinion.
Now, say I want to make a new particle? Don't modify particles.dat, instead make a new file with your new particle(s).
"But I need to link it to a hull object!" Modify the submarine file? No, again you don't have to in alot of cases. Add sounds to a sub interior, modify the original DSD? No, a seperate new file will do... It all depends on what specifically you have to change of course, but can be done for alot of mods. Just remember most of the naming semantics used by the original game are not required in most cases.
If everybody does that, there will still be conflicts but alot less, and using a good naming scheme (like <modname/description>_<version>_<optional stock filename>: MyParticles_1.0_particles.dat) can improve understanding of what does what. And mods can coexist without explicit need for version A or version B of another mod.
Obviously, this won't work for things like cfg's and campaigns etc, but it will help...
PS: I think you may even be able to overrule original content in a different file but I will verify this before I write down how. It may not work, and I don't want to be responsible for CTD drama's
