As a re-new horse owner (off and on since I was 7 years old), I can totally relate to what you're feeling. But unlike you, I've made some poor horse-buying decisions in the past, so my fears about picking the wrong horse were definitely warranted. (And every time I made a poor buying decision, it was because I allowed someone -- a friend or a seller -- talk me into more of a horse than I was comfortable taking on. Anyway ....)
My instructor -- who has owned and trained world champion appies for several decades -- recommended that I let my mare settle in for a week before riding her, and her advice was spot-on. For the first week after my mare came to the new barn, I groomed her and led her around the facility and let her get used to my voice. After a week, we just did a little ground work and then graduated to walk-trot stuff. We took it slow, and it worked out well for us. I wanted every "first" to be positive, and it was. Just something to consider.
Good luck and enjoy your new horse! Post pics when you get the chance!