Being gelded late wouldn't help the cause as he'll still be have some of that stallion behaviour running through him.
Don't go blaming the past owner saying he was doped up, this happens very rarely but it is such a common excuse when peopel get a new horse and it isn't exactly as it was when they saw it with the previous owner. Horses change depending on their environment, and they WILL be skittish to start with. Heck, I just got a 21 year old tb gelding for my dad to lease who a friend has had since he came ott. He was famous where he came from for being inanely quiet and nothing fazed him. Came to our place, and the first week he went absoltuely nuts everytime one of the mares walked away, he'd hype himself into a sweat and I couldn't touch him. Took 2 weeks and now he's completly settled, but as you can see, they Do change when you bring them somewhere different ;)
I also wouldn't be feeding him up yet, as what you have said is very typical new horse behaviour. He's been introduced to a new environment with new horses and people to adapt to, so of course he will be a bit spooky and on edge. Feeding him up with increase his energy and so you may be heating him up without meaning to. It's even good to have them a little ribby in those first few weeks while they adjust, then once they're happy with their new life, start putting that weight on.
I wouldn't be riding him until he's been with you for a good 2 weeks, letting him settle in and be a horse. When I get new horses in, I always just let them sit out in the paddock for at least the first week, only spending a small amount of time brushing and feeding them. After that 2 weeks, depending on how the horse is coping, I will start at least a week of groundwork and if that all goes well I'll start riding on the third or fourth week.
If you start riding too quickly, you're overloading the horse with new sensations and so often you will end up with a nrevous and tense horse on your hands for some time.
When you do start riding, put the horse on the lunge first. No matter how quite he was when you rode him at his old home, he will be nervy so er on the side on caution and lune first to get his brain into work mode.
Have to point out that getting off when he humped on you was a bad move, if you were terrified, someone else should have got on and ridden it out of him. If he scares you in these initial stages of owning him, he will work out very quickly that he can get away with being naughty and pulling the wool over your eyes and will start to challenge you even further until he may actually become dangerous.
As for your being nervous around him, don't ride yet. Get used to him and let him get used to you. Spend time with him on the ground, brushing is good but you also need to make sure his ground manners are up to scratch. Make him move around you, anywhere that you touch him you want him to move away until you allow him to stop. So if you push him on the shoulder, he needs to yield him shoulders until you take the pressure off. Same with his quarters. If he doesn't move, invest in a crop or dressage whip, and ask nicely with pressure from your hand/fingers, then if he doesn't react or leans into the pressure, a sharp tap where you are pressing will waken him up. You absolutely CANNOT let him walk all over you, especially as he was gelded so late in life.
Best of luck with him, would love to hear how you go with him and see some photos :)