Like most guys above suggested, it's not really worth the effort. In tonnage, they are not worth much. In combat, you are seriously outgunned until homing torpedoes become available and even then it is dicey at best. You can't outrun a destroyer on the surface, let alone submerged. On the surface she also easily outguns you.
In almost all situations, I apply the golden rule of dealing with destroyers: if not detected, outsmart her, don't engage her. If detected, go deep, go quiet and sneak away.
Less glory in that maybe, but you live to tell the tale.
In real life, a u-boat commander was supposed to sink tonnage, not engage the Royal Navy. Of course, if a target of opportunity presented itself, they'd try to sink it, but destroyers are not targets of opportunity.
Just my 2 cents.