While I love riding in halters, flat halters have a tendency to twist and pinch quite a bit. If you clip the reins/leads on the square cheek holes when you pull either the halter twists and prods them in the face. At a therapy program I worked at all the horses rode in halters. The reins would twist the halter out of place, sometimes poking the horse in the eye with the cheek pieces. Even rope halters have a tendency to twist and poke them in all the wrong places. She's found that putting her head down relieves whatever pressure the halter is putting on her face.
What I do when I encounter issues like this is video myself and look at exactly what the horse is trying to fix/avoid. Then alter accordingly.
Which is why if you really want to ride bitless, you need hands as soft as if you were riding in a strong bit, and you need a tool designed to be ridden in. So a cross under, a sidepull or a hackamore of some variety, depending on your riding style and preferences.
ETA:Also, remember, even if you're riding without a bit a halter can rub on their face and push against their teeth, which if they're sharp can really hurt!! So even if you don't ride with a bit this could very well be a tooth problem