I took him for a trail ride last Monday for a couple hours, all on gravel roads though I tried to keep him off gravel whenever possible. He had only been on soft sand and pasture. I had him trimmed Monday after our ride, but it was very minimal, just shaping and I wanted his hooves looked at because we had SO much rain and horses were coming up with abscesses and thrush like crazy due to the fact it would not STOP raining all month.
His feet are not tough. You have kept his feet soft by keeping him off the gravel and then asked him to walk on gravel for two hours. Yes, he's sore. And then you had him trimmed right afterwards. So his princely-soft feet got sore and then instead of a break, he may well have had what little callous he had removed. Now he's even more sore.
Plus he may have thrush? Does he have thrush? This can be painful as well.
He needs time to toughen his feet for sure. Don't make an effort to keep him OFF the gravel, you want to work him slowly up to be comfortable on gravel. He needs daily walks on gravel. Start with leading and short walks. Work him up to longer walks on gravel until he no longer has to "find his footing" on it. Then walk him under saddle on gravel.
As for his one foot that seems to be more tender than the others: it sounds like he is sensitive only when you are picking his that foot, not when you pick up the other front foot. If so, try to pinpoint an area where he flinches when you tap with the hoof pick. There might well be a bruise there. If you can't find one particular area, it might be thrush.
As for getting someone out -- You could call your farrier and ask him if he can remember anything particular that might be causing your horse some pain. Also, confirm whether (s)he thinks your horse has thrush.
You could post some pics of your horse and his feet. There may be other issues as well. So hard to know from a few words.