It all depends how 'broke' you want a horse to be. For a horse to be really broke and to be a really solid trail horse, it needs to be comfortable in the front, middle or behind other horses. It needs to be willing to go a different direction and should basically go anywhere it is asked by its rider. Any problems doing any of these things are flaws and are great 'big holes' in its training. If you are happy with that and don't care, then it is not a problem as long as nothing ever comes up where better training preparation is required.
When we start one riding out on the trail (usually the 4th or 5th ride on young horses) we start out by riding them by themselves. We ride the colt with an older horse and start out together for about 1/2 mile. We always start out with the colt in the lead because we do not know what we have if the colt is just following. Following is really no more of a challenge (or a training exercise) than a foal following its dam out in the pasture. A half mile out, the trail divides and we take different trails, usually at a jog. The colt will whinny, but I do not scold one for it. I just keep him moving on at a good trot. I do not ask him to cross water or go anywhere really tough for the first couple of rides. I set them up for success on an easy trail that should not cause a confrontation. I only ask what he is ready and able to do successfully. I am only interested in good forward impulsion on a loose rein at this point.
If everything goes OK, we meet up at a prearranged place and time. We ride together for a little ways and then split up again. Again, there is some whinnying but usually the colt is fine.
One of us will circle around to another trail and head back to the truck. Once I am headed back toward the truck, (always making a big circle and never just stopping and reversing directions), I will only let the colt walk. Even the first ride out they know where the truck is. They have a HUGE 'homing instinct'. If the colt gets in a hurry, he is taken off trail and must negotiate the brush and tall grass and rough ground. I never get in a pulling match with one. I may have to turn and go away from the truck for a ways, but I will insist on walking on a loose rein going back.
I think these first few rides out set the horse's attitude for future rides. I am very picky how they are conducted. I want a brave horse that trusts my judgement and goes anywhere I point his head.
By the 3rd or 4th ride, most colts are ready to cross water, gullies and go about anywhere.
Once they go on by themselves anywhere asked, we play the 'leap-frog game' as we call it. We start out together. One rider rides on ahead at a trot or lope (way out of sight) while the other rider 'tries' to walk on quietly behind. This often means the rider left behind has to ride off the trail in the rocks and brush or even turn back for a while to keep that horse quiet and walking on a loose rein. When that horse is walking nicely, it is bumped up into a trot or lope and that horse and riders passes right on by the first horse (hopefully without hesitation). That horse is ridden ahead and well out of sight or hearing range of the horse left behind. This routine is repeated until it is time to head back to the truck. Again, they are ridden in a big circle and go back on different trails.
For us, these are not 'pleasure rides' -- they are 'training rides'. It is worth all of the trouble in the long-run. You have a horse that rides anywhere with or without company, in the front of or behind other horses and are willing to leave a group and ride safely and sanely anywhere the rider wants to go. For us, it is just part of making a 'broke horse'.