The horse would rest back at the castle or designated safe zone for the journey or near the raid zone, after all there's no need to gallop there, a gentle trot would get you there and still have a fair bit of wind left in the horse. Like I said, the best way to survive is avoid contact, and any motor car creates noise, lots of noise, giving your position away to anyone on foot or on horseback. I think you underestimate the horse quite a bit, while it might not be able to get close to the carrying power of the big trucks, it doesn't need to carry around a great big load of fuel and its range is unlimited providing you can find water and grass which in my neighborhood is rarely a problem. As for speed, even loaded, a horse is faster than a standard zombie, if it's a fast zombie then there might be a few problems, but that's what I've got the shotgun or AK for.
Dealing with the anarchy is the next step, and another place where the horse might come into its own. Eventually fuel and ammunition will run low unless people start making their own bullets (which with 78 odd percent of the population gone is pot luck if the right people know the tools to use, etc) and making their own gunpowder, etc. So we will be back at the bow and arrow/crossbow and sword stage. The castle will defend well against those and hopefully by this time we should be near self-sufficiency and/or have enough manpower to put up a defence against raiders. Seeking out and establishing ties with other communities and making alliances between us to help protect our trade routes from raiders and any remaining zombies would be the next logical step, after all, we're bound to have something they want and they're bound to have something we want. From there, the UK returns to the fiefdoms of old.