Yah, I usually play with contacts on, now. I tried playing without it, and while immersive, it also makes patrols take a long time. I still run a patrol on occasion with them off, but it's not my usual play any more.
No doubt it can be fun tho. A lot like cams on/off. Both provide pros and cons.
With cams on, you can "see" all the stuff occur. Watch the torpedos move in, watch the big boom of the impact, etc.

With cams off, you are forced to sit there and wonder, perhaps pray a little. As stated, it's especially awesome when you can hear the depth charges going off, and have to wonder how close they are.