Congratulations on getting a horse.
When you ride your horse is down to you. Depending on how far a horse has travelled, I either ride the next day or the following. It also gives them a bit of time to familiarise themselves with their new surroundings etc.
As for feed, have you asked what feed he/she was on before? What sort of workload will he have with you? How much does he weigh? All of these will play a large part in how much and what you feed.
Whats your new horse called? Bet your very excited.
If you haven't picked out the horse you want yet I suggest you chose one that fits your level of riding. An older horse makes a good first horse. If you have little or no experience look at horses at least 7 years old or older. Also don't chose a horse just because it is pretty. I'm not sure a rescue horse is a good choice for a beginner unless its a neglect case and the horse has been well broke in the past. Take someone who is horse savvy with you when looking at horses. Do at least a couple of test rides. We trail ride so when buying our last horses we went trail riding for about 3 hours on them. Have a vet check done, there are some people out there who will drug a horse when someone is coming to look at it so it appears quiet and then when you get it home its a demon horse When we have looked at horses we tell the people to leave it in its pasture or whatever until we get there so we can see how easy it is to catch. That's a big thing for me, I hate a horse I have to chase all over the field and refuse to do it, so I had better be able to walk up to it. I guess that's a long enough post :) Good luck in your search.
Ok one more thing, don't rule out mixed breed horses. I had a mix that was a really great trail horse and I got her for cheap because I wasn't paying for her papers.
"Until one has loved an animal, part of one's soul remains unawakened..."
- Anatole France