Fishing, hunting, skiing, mountain bike riding, rock climbing... These are all mostly non-competitive (though of course they can be), don't have rules beyond e.g. Fish & game laws, but would all be considered sports by any reasonable definition, and are referred to as sports in everyday use.
Well, it's getting off topic, but:
Actually those are competitive sports and meet the definition of a sport.
Fishing for sport has been around for centuries and there are different rules for different fish and style of fishing. Even down to what's acceptable as a lure or bait. As a sport it's clearly defined as being different than subsistence fishing which has no rules beyond obeying the fishing laws of the where you are or commercial fishing and the laws governing it (subsistence and commercial fishing are not considered a sport).
Hunting is much the same (type of game, weapon, etc....). In some cases even more "nit picky" than the fishing. Like fishing, it's often based on the overall outcome of the season, although it can also be based on the results of a day, week or whatever time period the competition calls for. And like fishing it's differentiated from subsistence hunting, etc.....
Bike riding is much like horseback riding. There are certainly cycling sports, but simply riding a bicycle is not a sport.
Rock climbing as a sport does have rules (even judges), but there are also people who just do it for fun and don't have to worry about all that. And there is also the competition based on climbing the highest or toughest (not as many rules).
Point being that when done as a sport they do have rules and the people do compete. E.g. Getting the largest deer or the one with the most points that year.....catching the largest Bass (providing you don't violate certain requirements....you should see what they go through to be able to qualify for catching a world record bass....it's not just catching the biggest fish, but also not violating any of the rules).
Of course you can also list mind sports such as chess, scrabble and other games. Not very physically demanding, but still a competition with rules.
I'll go back to an earlier statement. If riding a bike, riding horse, etc.... is a sport than the vast majority of what people do is a sport. E.g. There are walking sports, but that doesn't make taking a walk a sport. There are probably more sports using an automobile than there are using a horse, but that doesn't make driving to the store (or on vacation) a sport.
There are requirements for something to be a sport. If there wasn't then everything anyone ever does is a sport. My ex was a world champion at the sport of sleeping