I agree with this for myself, but have never worked in a show barn. That being said...
-OP- there is a movie where the dad says to the son "Don't let anyone, even me, ever get in the way of your dreams"..... Build a plan- literally write it down so you have guidelines when you are down or when you want to give up. I encourage you to whole heartedly follow your dreams. YOU can do it!
Do it in small chunks. Things appear so much harder when you look at them as a whole. Looking at my major uni assignment right now, and as a whole, there is a HELL of a lot to do, I feel stressed, rushed and panicked, 4000 words in 2 weeks with tables, diagrams and charts. However, using my normal technique of breaking it down into bite sized chunks, I can get the job done with the stress level.
Write down your specific goal. What discipline do you want to specialise in? Do you want to be a professional horse trainer, rider trainer or competition rider?
Then work back from there, the steps that are needed to get you to that point. So the first one may be perfectly your own riding goals. You say you always look down, so the first may be LOOK UP. Focus on your position, once you perfect your position then start the next goal, maybe working on collection, getting a perfect form over a fence, etc. Build it up in bite sized chunks. You will feel a much greater sense of achievement when you get to mark off each goal, you'll see yourself getting closer to that ultimate goal and feel more inclined to keep working at it as you reach closure on each goal you have set yourself.
Don't make your goals too big, if you have to work too hard and find them unacheivable in one step, break it down into two smaller steps.
best of luck. Don't let people tell you it's impossible. But DO listen to the advice given. You have to go into it with as much information as possible and take off the rose coloured glasses to realise it is by far a horrendously difficult journey.