Well, I have been looking around a bit and I am pretty sure now, it is the starforce problem. The starforce version on the SH3 DVD does not work with XP 64 bit. And because of that, you get the error message.
Have a look at this one:
http://www.subsim.com/radioroom/showthread.php?t=160118
Read it carefully, all information is in there. Find the removal tool, google the other archive thing mentioned there.
SH 3 should run after that. Be aware that other old games you are going to install could use Starforce as well...
A better option is to grab a fresh DVD without starforce. It is less than 10 $. Actually, it is 3.99 $ through SubSim Amazon store today...