Assuming that the bit fits and his teeth are not hurting him, maybe he just needs more time with the bit. I left my gelding to wear the headstall and bit (no reins) so that he learned to carry it happily. Just left him wearing it while tied for a couple of hours every few days while I was working around the barn. Two lessons in one, patience while standing tied and carrying the bit.
Just one more thing to check out - does he have ulcers? Teeth grinding even when not being ridden is one of the signs of ulcers.
And to get rid of your stress, spend time grooming and touching him, before you get on. You may not even need to ride - just let yourself calm down and imagine good rides, don't think about your stress inside your head. Put a picture in your head of what you WANT to be like, calm, in touch with your horse, go for baby steps and a little bit of progress each day.