Lol! I have my mares since they were yearlings, so it is what it is. :) I feel more comfortable english than western (even though I started them in western saddle). BTW my qh did a better job at the shows than me (I'm getting too high and can't do much with my nerves). However my paint was not shown because of the reason you just stated: she was not ready and is too nervous horse. So I much rather put a better foundation on her.
If your horse is spooky I'd just keep working with him until both of you feel confident (and 4 years old is still a baby). If you are nervous the horse (especially the younger one) will feed up on you.
Good luck! I'm sure you'll start showing successfully, just don't be in a hurry and try when you feel you are ready!
"Life isn't about waiting for the storm to pass: it's about learning to dance in the rain..."
"When we are no longer able to change a situation - we are challenged to change ourselves."
"How people treat you is their karma; how you react is yours."