I would think that if you wear a helmet, then you would naturally want to wear a vest...I don't wear either so I can't speak to that as a reality, just making the "logical" progression.
I am the only rider at my jumper barn that wears a vest. Most are opposed to them, some are curious... a few plan on getting one like myself. I could go with/without a vest but knowing I have some added protection makes me feel better, especially when schooling over new jumps.
The hardest part isn't finding what we need to be, it's being content with who we are.