What I do when I encounter issues like this is video myself and look at exactly what the horse is trying to fix/avoid. Then alter accordingly.
Which is why if you really want to ride bitless, you need hands as soft as if you were riding in a strong bit, and you need a tool designed to be ridden in. So a cross under, a sidepull or a hackamore of some variety, depending on your riding style and preferences.
ETA:Also, remember, even if you're riding without a bit a halter can rub on their face and push against their teeth, which if they're sharp can really hurt!! So even if you don't ride with a bit this could very well be a tooth problem