Horseback riding should be fun. You have quite a way to go before this is a fun horse. Trading is certainly an inviting option. You can get a new horse. You can't get a new neck or brain.
If you are going to keep him, he needs to learn whoa. I would work him daily for two months in an arena before I even considered taking him out again.