Recently someone ‘admitted’ to me that her horse didn't always come immediately when she called him. She felt that this was a sign that their relationship wasn't as good as it could be, and was relieved and surprised that, when she explained his behaviour in more detail, I thought it was a sign of a healthy relationship and good management.
Of course it seems nice when your horse is waiting for you eagerly at the field gate, or comes galloping up the field as soon as he hears you coming. My horses still come up to the yard when I arrive, but not in a desperate hurry, and mainly because the first thing I do is provide food! They are all perfectly willing to spend time with me; sometimes they approach me; at other times they are more interested in playing, grooming or just hanging out with each other and I need to ask them to come to me.
This has been reinforced recently - I'm currently at home on 'box rest' following an unplanned operation. I probably won't see my horses for a month; in the last 10 years I've never spent more than a few days away from them. So it is a great relief to know that they don't depend on me - although I think they enjoy my company, many of the good things in their lives have nothing to do with me, and I doubt that they will suffer from my absence.
Clearly, if your horse is heading for the hills every time you show up, or retreating to the back of his stable, there is a problem. But if he simply takes his time coming when you call, or you occasionally have to go and fetch him, it may simply mean that you've got his management right and he's happy!