Probably nothing to do with your graphic settings, but rather with your RAM capacity...
Have you installed Large Adress Aware application to boost your RAM?
Have you installed the 4GB patch?
I run both for mega mods such as GWX or even TWOS
I highly recommand H.sie patch (for GWX) wich contains the 4GB patch inside, plus many other interesting features.