Lol I wouldn't worry at all. :3 I was used to riding the draft breeds then bought a 14.2 pony. XD I'm 5'5/tinier build and think I'm huge on him (Just used to bigger) but so long as the both of you are comfy and are just in it for the nice ride you should be good and shouldn't have anything to worry about. :)
I wouldn't think so. I'm 5'6" and a lot of my QHs are little stout 14 handers. My sister showed a 13hh quarter pony when we were growing up, she's 5 ft exactly. They were a perfect match. As long as she has the barrel to take up your leg I wouldn't worry about it at all.