As the other fabulous posts have noted- punishment results in the reduced performance of a behaviour and reinforcement (reward) results in the increased performance of a behaviour. This is not just true of horses, its true of any species that learn new behaviour, including humans.
Irrespective of the label (correction, punishment) if the stimulus causes the horse to stop performing the behaviour then that stimulus has punished that behaviour.
The scientific evidence to date suggests that we have no more than 3 seconds and possibly less to punish an unwanted behaviour. This is because for the horse to know what behaviour caused the bad outcome (the punishment) the bad outcome has to follow during or immediately afterwards. Otherwise the horse might do something else and then get the bad outcome. It will think that the something else caused the bad outcome, not the behaviour you actually want it to stop doing. This is known as the Law of Effect (Watson, 1919). Punishment only tells the horse what not to do, it doesn't help the horse do anything else instead. That is one of the biggest limitations with punishment.
As others have noted, "punishments" such as tying the horse up for hours after unsaddling it after it bucked you off, endless laps of a round pen, withholding food after a trail ride etc etc, will not be effective at reducing the unwanted behaviour. They fail the 3 second test. They may seem to make the horse more pliable because its tired, hungry, thirsty etc but it won't have learned anything useful.
All the scientific evidence to date suggests that horses cannot reason and that they don't have any insight into their behaviour or ours. When we 'punish' a horse for doing the 'wrong thing' we are assuming they are much smarter than they are and that they are moral beings with a knowledge of right and wrong. This is unfair to them and reduces our effectiveness as trainers because it blames the horse.
Thinking of horse behaviour in terms of whether it has been reinforced (repeated) or punished (reduced) can give us a great deal of clarity in analysing training success and failures. Its also much simpler than getting anxious about whether you are punishing or correcting your horse.
All the horse wants to work out is how to get good outcomes and avoid bad ones. As long as we and our horse are in agreement about what constitutes a good and a bad outcome for them, training should become easier for us and much fairer for them.