Originally my temporary electric was only 3' high. Now I have permanent wooden posts and its 4 foot, 4 strand electric fence tape.
It really depends on the horse. One of mine, my appy, is so respectful of the fence, that even if I take the fence down, she will just stare at where the fence was, and will not go through until she sees me go through safely, or I walk her through to show her. My QH would barge through any fence if it wasnt electric whether it was 3' or 17'! I would suggest it be electric if its only 3', which I am sure it is if there were cattle/sheep in there before. Ideally, 4-5 feet would be best.