The problem here is your own insecurity.
It's your horse - you get to decide what to do with her. Period.
If you want to hire a trainer - then you hire a trainer. As in $$ for XX sessions.
Right now, I feel like you are hem-hawing around waiting to see what OTHER people think, or do, or say, or might say...
I feel like I'm not getting anywhere on my own or maybe I am...I don't know. I was wondering...
And so instead of being decisive and making a plan to achieve the goals you want - you are second-guessing yourself and going nowhere. And it is that second-guessing and lack of confidence that is inviting the comments about what others *might* be able to achieve in your stead.
Make a decision. Own it. Have some faith in yourself.
Then you will find that such comments do not bother you - if they are made at all...