I have always gone in on the eastern route.
Travel on the surface as far as you can (do the last leg in the night), I would say at least until the archipelago is north of you, farther if possible. You will meet destroyers; dont shoot, let them pass, then surface and go on.
When you get close (how close you can go in on the surface depends on sea state, time of day and your luck), stay submerged and go in on silent running, slow speed. Prepare to stay submerged all the time from now on.
Most of the time you will be at periscope depth because the bay is so shallow. Have short glances at the destroyers you will see to determine where they are going. Most will pass far from you. If one looks like it will come close (e.g. AOB 20° or less), stop and let it pass. Dont shoot.
There is a submerged barrier from the eastern shore into the channel (probably the western too) where the big bay starts. If you go in at the center of the channel, you wont touch it; else (if you sneak along the eastern coast, which I do) be *real* careful and look ahead through the scope... stop or brake (back slow) when you see the barrier and go along it to the west until it ends. I would say time compression should be x4 at most when you approach the fence, or you will be unable to react quickly enough. Hitting the barrier sounds like running aground, so that might be what happened to you?
Once you are in the main bay, you can either go for the harbour directly or sneak to the right, where you can hide in the channel between the two islands. The ground is real shallow there, so be prepared to go up almost to the surface (they will still see less of you if only your tower is out of the water). You can surface in the narrow channel to replenish oxygen or even recharge batteries a little, but I would not recommend staying too long. Obviously, doing this in the night gives you better chances. I have never been detected there, even during an overnight stay
Then on to the attack, and back home. Remember, stay under water all the time and peek out with your scope only occasionally to get your bearings.
The key factor here is RL patience. You will go slowly, and you will get sonar messages all the time (= TC down to 8), so you have to work TC (time compression) a lot... progress will be slow in any case.
Stay at slow speed, stop when you think someone might detect you. Remember, there are shore batteries that will see you & shoot at you, so dont get careless near the shore.
When the attack is complete and you dont feel like going all the way back, there is a good chance you can surface in the middle of the main bay and just go home with the =ESC= key. No need to reach international waters, just stay undetected.
And prepare to have done all of this in vain, there is no guarantee any big ships will be in Scapa Flow. Happened to me more often than not.
Hope that helps, sorry if this is a tad long