Sounds like it happens to all of us. Both my soundmen are PO's and experienced and have problems with escorts getting lost in my baffles during convoy actions.
Times like that I man the phone, then back to the map, then back to the phones. What is cool is hearing the splash of the depth charges then the whine of the acceleration. As long as they are not too close.