I play at 85% realism with IUb. Haven't been able to make it past early 1943.
I have been killed at depth by destroyers. Killed once (and instantly) by the very first DC volley when I dived too late, got down to about 75-85 meters (my last career). But far more often, I get killed by airplanes. Two careers ago, a flying boat dropped a bomb smack on target, just as I was almost below the surface, and that was that.
It is true that you can go pretty much unscathed in the early war. But that's what is supposed to happen in this sim. By the end of the war, it gets extremely hard, and most U-boats (the vast majority, in fact) did not survive.
For me, this is the real challenge – to survive the war. I figure if I can survive it once at 85%, I will then crank it up to 100% realism and see how it goes. (Have to wean myself off that external camera, though.)