I have the same problem. Do you have a Nvidia card? When i had my ATI card I had no problems with setting the AA for SH3 but with two separate computers both with Nvidia cards, changing the AA just leads to a black screen. I tried the latest drivers and no luck.
I think Nvidia is a better card overall, but no luck with SH3.