Really you should always be sitting, standing and walking with a straight back even when not riding. If you've been riding for a long time it can be easier to get a straight back.
I'd say I have very good posture due to drumming as well as horse riding.
Shoulds scrunched up, put back then relax them and put them down.
Up, back, down.
Make sure you're holding the reins at a good length as well. It helps.
Position yourself on the horse, straighten your back and get someone to put a riding crop through the gaps in your arms passing your back. If you slump your back simply the crop will break or bend.
Practise as much as you can.
Horses heal hearts. - Gleekful Gleeks.