I ridden my horses with just halters on a regular bases since the later part of the 70's when I gave up bits and worked cattle with reins attached to a halter. Quit working cattle in 85, but enjoy not having to deal with bits. Horses sure don't seem to mind :)
It's always fun watching people when you ride up (or by) with just a halter (like I did today). They're usually shocked to see someone riding with no bit and the horse is under control (like a bit really gives control :) ). Makes for questioin and answer time and good for meeting people :) . I do use two leads or clip on reins though so I have one on each side. If your horse is well trained for neck reining you don't need a halter. Just wrap a cord around it's neck and you can neck rein with that. But that does tend to really freek some people out when they see that so I won't advocate it except around home.