Luck plays a big part in this, a heavy cruiser generally needs at least 2 or 3 torpedoes, sometimes 4 or 5 depending on where they're hitting. If you can hit them right under a turret set for just the right depth a single torpedo can set off a secondary explosion in one of the main magazines, starting a chain reaction so it's gone with only one hit. Same thing with the deck gun, each ship model has a bunch of damage zones, some more critical than others.
If the lack of visible damage is constant, check your Options/Graphics settings - just below the resolution is a ship 3D damage slider, all the way to the left is PG rating, all the way to the right shows all the gory details.
Want a more effective gun without getting too unrealistic, try this;
http://www.subsim.com/radioroom//dow...do=file&id=517
Several options for increasing torpedo damage;
http://www.subsim.com/radioroom//dow...o=file&id=4380