OLC GUI won't make your range calculations from the periscope any more accurate, I'm afraid. But you really don't need any more accuracy than you have. If you have accurate target course and speed data then the range does not need to be super-accurate in order to get an excellent firing solution.
The reason you're having difficulty getting an accurate course is because (if I understand your post correctly) you're trying to get it in five minutes. IRL commanders spent hours (sometimes days) tracking a ship/convoy in order to get accurate course data, and this is what works best in SH3 also. Spot a ship/convoy, mark it's position and approximate course on the map as you are doing now. Then track it at the edge of visual range for... lets say 50km. This is quite arduous and requires constant adjustments to your course to make sure you don't get too close or too far away, so if you're feeling lazy/lucky just flank it and position yourself 50km ahead of it and wait for it to re-appear. Either way you mark it's position again once it's moved 50km. You now have two marks separated by at least 50km, which means each mark can be innacurate by a couple of kilometers and you'd still have good course and speed data. For even more accurate data go another 50km... but beware of the ship/convoy changing course!
When such manouvers are impossible (because the target is moving too fast to be flanked, or is heading into dangerous/shallow water, or whatever) you can use the tools in the OLC GUI to get an immediate firing solution without any prior course or speed data, provided you can get into a firing position (and the mod is "historically accurate/realistic" as well).
This video shows that with accurate course and speed data gathered over a long time/distance, the range doesn't need to be that accurate to score perfect hits.
HTH
OLC <- (that's me
)