Actually, I got a little wrapped up in U-boats with my thinking and forgot about one decisive advantage that the US submarines had from quite early on in the war: effective, working SD radar. The SD radar honestly does do wonders, and while you still need to react to its signals in time to avoid a threat, if you dive as soon as it makes contact, the dive time is usually a non-issue. Even early SD radar will generally allow you to get under before the plane actually gets close enough.
SD radar, not AA guns, is a fleet boat's real plane antidote