Running gun battles are always a bad idea. The reason they 'saw' you at three kilometers is that they knew exactly where to look. You had already fired on one of them, so the AI on the others was in 'Alert' mode.
Instead of wasting your torpedoes on the escorts you probably should have just targetted the merchants and been done with it. Also, merchant gun crews were better trained than you might think, especially as the war progressed. Deck guns are for sinking helpless victims, not ones who can shoot back.
Here's a very good discussion of the subject.
http://www.uboat.net/forums/read.php?20,64443,64448
[note] This is actually a response to a question on that forum. The original question and other answers are also interesting.