It has nothing to do with your system RAM, it is your video card RAM. Your Video Card has a maximum of 256MB on it I am guessing. Ultra Textures really require 1GB of video card memory or more. Less than that and it swaps out to your system RAM over a slow BUS. This may be the cause of your strange artifacts since over land, it has to load more textures yet doesn't have enough storage to do so.
Besides, smooth frame rates in X-Plane are not enough. It's modeling requires you maintain extremely high frame rates due to the way it calculates flight. Drop below that and it is no better at simulating flight than MS FLight Simulator. X-Plane actually calculates airflow over the flight surfaces of the aircraft instead of just giving you a fake flight model. This require high FPS to achieve.
-S
__________________
 
|