Interestingly U-boat hulls seem to be a lot tougher than you'd think. This is from an Admiralty Intelligence report on U-boats published in 1944 in regards to machine gun and cannon fire from aircraft:
"The blister tanks are probably vulnerable to .5 machine gun fire, but even very severe damage to blister tanks will not sink a U-boat. The only vital target for cannon or machine gun fire is the guns' crews and bridge watch."
"There is, however, a case on record in which leaking oil from saddle tanks, penetrated by an aircraft's machine gun fire, was a contributory cause in the subsequent sinking of a U-boat by a surface vessel."
It makes no mention of machine gun or 20mm cannon fire being able to penetrate the pressure hull.
I have however read of one U-boat being forced to return to base after having a large hole blown into its conning tower by a hit from the 57mm cannon of a Mosquito Tse Tse, rendering the boat incapable of diving.
You can read the rest of the report here... it's very detailed and interesting stuff.
http://www.uboatarchive.net/CumulativeEdition.htm
PS As for Elco boats in game, I once sunk three of them with the AA gun in the GWX single mission 'Evacuation'. They didn't really cause me any problems in the mission, although in a campaign I'd probably just dive and avoid them, as I wouldn't want to risk losing one of my bridge crew and they'd just call in reinforcements if any enemy surface or air units were nearby. So not really worth it.