He moves very well and has nice movement, he doesn’t move under himself as much as he could and seems to be a little stiff in his back end. He doesn’t like to stretch his legs back when you pick up his feet and cant balance on 3 legs very well if at all ( for example having one foot lifted for over a minute for the farrier is difficult). He often times slips in the hind end and trips,
That first comment above seems to contradict the rest greatly. Have you had a bodyworker or at least vet to check him over? If so, that's where I'd start. If he's been a jumper since ?? I would guess there may hock, stifle, &/or sacro problems that affect him. Whether or not anything can be improved, a good bodyworker may give you some important advice on how best to manage him.
I was told he has four shoes because his back end is weak and they put back shoes to “support him”,
I don't get that concept.
I am wondering if I should pull my horses back shoes to save money and the foot, or leave them on. I am afraid if I pull them he wont move as nice, or be more reluctant to move under himself. I also want to be able to ride various terrain and not have to worry about sore feet or any of that. ...He has good hind feet, but I dont want to ruin them by taking the shoes off..
Well, addressed in order of above comments, firstly I wouldn't do this just to save money. Assuming this horse isn't an expendable commodity that's going to be 'thrown away' when he is broken, I think that whatever is best for the horse tends to be best financially too, in the long run.
As for not 'moving nice', it doesn't sound like he is anyway. Shoes shouldn't make him less reluctant to 'move under himself'... assuming he can do that at all anyway.
Riding on various terrain... depends on many factors, esp on environment & terrain the horse lives as to whether they need hoof protection &/or support in some situations at least. Most horses will need some protection on rocky trails for eg. Conventional metal rims provide little if any(depends on terrain) protection & no support to the underside of a horse's foot. These days there are many different types of hoof boot, that mean there is something suitable for almost any horse in just about any situation. They are also generally more economical, only needing to be used when extra protection is required, rather than 24/7, needing to be redone 6 weekly.
I am not dead against shoes, but IMO removing them will not damage feet(except in rare exceptions), however keeping a horse shod long term without respite does. At any rate, you will need to do your homework, learn the principles & factors that affect hoof health & function, learn the pros & cons of various approaches and ideas, and make up your own mind about what's best for your boy &/or you. The links in my signature will hopefully help you get started on that.