Well this could be a few things at play.
First: You might not put a big hole in the ship, you just cause big cracks or you put a small hole in and they patched it up. The cargo could be the reason but the game doesn't simulate this. Contained flooding could be why they list or settle but don't sink. It could be a bug in the game.

But from what I have seen of GWX, whats really happening is the fact that flooding and not damage is what sinks ships. Try to hit the bow while they move, the force of the ship moving will press water into the ship. After that they should be slow, take your time and spread out the damage to both sides of the ship and both ends. If you see them in a convoy, take your time to line up a shot that will disable them if not sink them then wait for night and then use your deck gun from range while moving fast and turning. Take out their guns, if your movements are fast and random you should have enough time to destroy their guns, use manual targeting of the deck gun.
If you wish to use just torpedoes then as I said before, spread out the damage use magnetics to maximize damage. If the ship isn't moving aim for cargo hauls or other large rooms, your looking for low part of the ship most likely the bottom deck somewhere fore of the stack tower and superstructure.
But if at all possible, use your deck gun to start fires and do damage to the deck, top cargo, cabin, and all manner of other structures. This should reduce the rate they repair their ship and pump water out. That is why they had deck guns...to finish ships off.

Good luck.

The tanker MV SAN DEMITRIO was one of these 'tough cookies', in 1940 pounded by the 'Admiral Sheer', she was abandoned in mid atlantic, but was eventually reboarded, patched up & sailed home. She was sunk 2 years later by U404.
http://en.wikipedia.org/wiki/MV_San_Demetrio

It was made into a pretty good film as well:

http://uk.imdb.com/title/tt0039797/