One of two things or both could be in play here. The first is fescue is not a yummy grass that horses willingly scarf down. I understand why you planted it. It's a hardy grass that will easily grow in all conditions and in all soils including clay. It is a rather nutritious grass providing a decent protein level compared to others. Why would your horse eat the not so good tasting grass out in the sun when someone will bring them hay if they wait long enough? Is your horse out to graze overnight when the bugs and sun go away?
The second point has different opinions depending on who you talk to. A horse does not eat where it poops, because it makes the grass taste bitter. When you drag your pasture and spread the manure everywhere, it makes all of the grass bitter. It also spreads any worm eggs all over the pasture where your horse can't avoid them. Spreading the manure only kills eggs in hot, dry areas. Any area that has humidity, this practice is a moot point since the humidity keeps the egg casing from drying out and dying. I live in the midwest where there is a ton of humidity.
So not so yummy grass that has manure in the soil, and hay brought to the horse does not surprise me that the horse is not grazing. I have fescue in my fields, and the horses eat everything but the fescue. However, if I bale those same fields and feed it during winter, they eat it because that's their only option. If I were you, I would make sure the horse can graze at night out of the sun, and slow down or quit delivering the hay.