I use GWX with a 20 km Environment mod. However, I would greatly recommend the Vampire Nightvision Fix that is currently in beta-testing.
Once completed, it should allow for much more realistic visual ranges using the original SH3 settings, or you can tweak to suit your desired behaviors.
The commonly quoted figure for spotting a ship in good light and weather is 8 km. I assume this short distance means seeing enough of the vessel to make some type of identification. To see just all or part of the superstructure, that distance would increase to as much as 25 km.
Spotting the smoke from the funnels could probably be done up to a distance of 40 km for oil-fired vessels. For coal burners, that distance can be as great as 80 km. This is assuming the vessel is getting weigh on at top cruising speed, producing more smoke as it burns fuel at a greater rate.
The real trouble is not everyone seems to agree on what is considered the definitive standard in these distances. Some argue there is no way to spot a mast at any distance where the rest of the vessel is hidden by the horizon while others have claimed it can be done, based upon sources drawn from actual u-boat logs.
The radar and sonar ranges used in GWX seem to reflect reality well enough, though I do think the enemy radar can sometimes be too adept at spotting your tiny conning tower in the early war years, but that's probably just me being prejudiced against those ASW chaps.
Keep in mind that the SH3 engine does not model sensors accurately. Enemy eyes, optics, RADAR, ASDIC, and hydrophones are in no way impaired by land or obstacles, so hiding on the leeward side of an island or beside a sunken wreck only means they don't have to worry with chasing you down while you're sitting there with the false security that you're hidden.
In the end, it is up to you to decide what feels more realistic in the game. I say this because you must remember the game engine does a half-ass job of simulating the real world so real world values may not offer the results you're seeking.