Determining AOB was more of an art than science, real sub skippers practiced using tabletop models with protractors and odd stuff like that. Again, one of the main reasons I use auto targeting instead of manual is because in real life I would have had a whole plotting team to do all the math for me, and it's just too much tedious work for a game. It's supposed to be entertaining, and I am not entertained by math problems.
That said, I would advise using training wheels - use auto targeting but take notes and screenshots to use it as a learning experience rather than loafing along with your mind in neutral.
AOB illustrated (with the AOB dial moved over for a smaller pic):
In this one the relative bearing FROM ME TO HIM is about zero, the AOB (Relative bearing from HIM to ME) is about Starboard 42.
Relative bearing here is about 355 (close to the same) but since he's facing directly toward me, the AOB (again, relative bearing from HIM to ME, not the same as from ME to HIM) is about zero. The "mini TDC" part of the scope view shows YOUR SUB in the lower dial, and the target in the upper dial, which gives you a tactical picture of what's going on.