Welcome Aboard, jbruns29!
I'm not sure how much either game costs on Steam, but on disc they are so cheap that buying both is an option. While SH3 runs better without any mods, there are so many great ones available for it that you really miss a lot running it without. This means that you really want mods no matter which you choose, so there isn't as big a difference in that respect as it might seem. Yes, SH5 needs the mods to make it run correctly, but it is well worth the effort.