Just don't think about it too much, I'm pretty sure it's bound to become a natural instinct eventually and you'll have it nailed down, as long as you know the roads and which way.
Yeah, Fa hit the nail on the head with what I wanted to say. I failed my permit test three times too. Started taking it in May 2013, didn't pass it until August 2014. Just keep trying and you'll get it eventually.