Well, it's getting off topic, but:
Actually those are competitive sports and meet the definition of a sport.
Fishing for sport has been around for centuries and there are different rules for different fish and style of fishing. Even down to what's acceptable as a lure or bait. As a sport it's clearly defined as being different than subsistence fishing which has no rules beyond obeying the fishing laws of the where you are or commercial fishing and the laws governing it (subsistence and commercial fishing are not considered a sport).
As for the horse doing most of the work in riding, would anyone like to show that it's not true for most riders, most of the time? Sure, it can be exercise, just as you could get a workout playing golf by running from hole to hole. But most people still prefer to ride in motorized carts