Congratulations on your first ever riding lesson! That is really exciting. It's normal to be really sore at first and even your hands will be sore. Riding is very athletic, make no mistake about it.
Gloves are good. Also, when you are just standing there listening to the teacher, be sure to give the horse some rein so it can relax. Plenty. If you want it to stand still and it is standing still, then you need not "talk" to it anymore with your reins, so let them droop a little. If you keep them "on" (pressure on his mouth) all the time, it's like driving with the brakes on all the time; you'll wear out the brakes and soon they'll mean nothing to your horse.
Be patient. There is a LOT to learn about riding . It takes YEARS to become a good rider, but you'll get comfortable at it in a few months.