Yes, they are wrong about him being too old to start barrels. In fact, if he has a good basic training background (leg yielding, lead changes, etc.), that is half of the battle with barrel racing. Also, I think it would be better to start a little older rather than younger with a horse because it is sooooo, sooo, soooo hard on their legs physically (bones & tendons).
It might be a good idea to get involved with a trainer if you want to learn barrels. Some things to keep in mind regardless: you should not immediatel expect to do a full speed run on the barrels - wait until both of you have the pattern down well (pacing, positioning, balance, etc.). You should spend most of your time working through the pattern at a walk, trot, and canter. Save the full speed runs for at the races and maybe one practice run before each race. Horse get burned out mentally and physically when they are RUN full speed through the pattern too many times in a row. Also, be sure to mix it up with him...keep up that trail riding to give his mind a chance to calm down and relax.
It is quite possible that if he was involved in roping in the past, he has had some experience with barrels, but if not, it is really pretty easy to teach them and get them "patterned", especially if he already knows the basics.
As for the speed thing, barrels are very high adreniline, and as your horse becomes patterned, be will begin to associate barrels with running, and in time, you will have to work hard to keep him from running the pattern. That is why you will want to spend most of your time at the walk, trot, and canter rather than run, run, run. Of course, some horses have more natural affininity toward that kind of sport than others, but they really do get that barrels means run. I would not worry too much right now about adding tons of speed - it will come as both of you get better and you are able to add more energy to your runs.
I would encourage you to keep doing things other than barrels, and keep working on the basics (soft on bit, supple, etc.). I can't tell you how many horses that I have seen at rodeos, barrel jack pots, and gymkanas that have their heads straight up in the air, pushed hard against the end of a tie down, bucking, rearing, dancing in place, and so on. You will have to work hard to keep your horse level headed.