Is there a possibility that you are causing him to be wary.
In my experiance, many times the rider is telling their horse that there is something to be scared of by tensing up when they see a large veichle or scary object.
If you feel yourself getting tense in the saddle - try to relax .
When you were out the other day, did the other horse get frightened first - and make your boy frightened as a result.
Keep him going out with a lead and try to keep relaxed when there is traffic around, focus on breathing slowly as this will relax you and transmit to your boy that there is nothing to worry about. Keep your hands and legs relaxed - don't change the way you ride. If you tense your arms and legs in preperation for him to be silly - you are the one who is telling him that there is something to be silly about - not the traffic
It's got nothing to do with him being an Irish Draft - I've ridden a 4-5 yr old out myself , Samson had a huge problem in that he had never been away from his mums side - but once he got over the initial shock of her not being there he was brave and bold in most situations and only once gave me cause for concern ( he had a cow phobia , one day we met a herd and he was not pleased ! )