Yeah, stock game bug, they're not really looking for you (not sure if they really are hunting your rather handy distant guided decoy though). I think they just become somehow 'associated' with your movements.
It can hapen with friendlies too; I once had an armed trawler (or the like, I forget now) bravely follow me all the way (at a distance) from wilhelmshaven to scotland!