most of your air dropped munitions were either 500lbs general purpose bombs, High explosive bombs or 1,000 lbs HE bombs OR even air dropped depth charges set for 20 - 30 meters.
A direct hit on the deck or conning tower with even a 500 lbs GP bomb would be devestating to the U-Boat... if the boat was not destroyed all together it would be seriously damaged and perhaps even crippled to the point of returning to port.
In many cases - and something that SH3 simulates poorly - is that U-boats were often damaged so badly in air attacks that the boat could not submerge safely. now they are sitting ducks to other aircraft and surface ships. You are correct in your assumption that a single direct hit from a DD's gun could prevent the boat from diving.
as far as the difficulty goes....
remember - U-boats are designed to present a low profile when viewed from the surface so shooting at one from a fairly long range wouldnt likely produce hits on the small target. but from the air- an airplane diving on the U-boat at 45 to 60 degrees - the boat is a large target and quite easy to strike her.
later in the war (january 1943 and later) allied weapons were better - allied pilots were better and many surface ships were equipped with radar and had radar directed guns which could be fired accurately even at a periscope from long ranges. But early in the war, even if you surfaced at medium range near a destroyer it would take him a few trial shots to hit you as range was more or less calculated by eye-balling it.
difficulty of the escorts also depends not only on the year you engage them, but also the mods you are using. for example with GWX from 39 - 42 i find it pretty easy to dive to 150 - 180 meters and escape detection and attack from escorts... but after intel had been found on several U-boats the allies learned how to better locate U-boats and what their true depth capabilities were. by mid 1943 i have found that the allies are using accurate sound location even down at 240 meters
making escape next to impossible!