If you sink a neutral ship, you will lose renown. If you sink multiple neutral ships, there is a possibility that Kriegsmarine may hunt you down when you return to the port
Except that you cannot change the history of war. Sinking neutral ships won't cause some nations to declare war or sinking all the British or US tankers won't cause Great Britain to starve crude oil and lose the war.
If you are running GWX (I'm not sure if this option is also available for the stock game), you can press F1 and check which nations are at war with Germany from "Enemies of Germany"