That's now five years I am living in the "inside" side of the door, so I get to understand it pretty well now. Although I spend a lot of time on the international french TV5 channel so I don't forget my original language...
My main problem in the beginning was that each region has its own dialect - germans from Hamburg don't understand germans from Munich - which is very different from what you learn in school.
Like one of my german friends told me in the beginning : The main difference between Germans and foreigners living in Germany is that foreigners speak German...