I have been to Iceland in June for 8 days and of course went for a horse ride 1 day and it was amazing.
Icelandic horses are amazing, fearless, comfortable to ride and strong. Even though I did not know the horse and we were just in 2 people out (me and the guide) I did not feel unsafe with the horse not even for 1 second, and Lolly was just super cute <3 . And the view!!! WOW, no words.
Iceland is all beautiful but if you want to go take at least 8-10 days and even though the south/south-east area is the one with most attractions, I would suggest you to go in the north or west part of Iceland. Way less people, especially for horse back riding. I went to https://saltvik.is/horse-riding/
and I think they are really well organized and open with schedules etc, we arrived without booking and a girl was immediately willing to come out with me (it was not high season so they just opened to tourists). They also have a guest house in case you need it. The staff, generally in a lot of places, is not from the country. Most of them (in my case, all the riders/guides I met) were foreigners living in Iceland for 7/9 months a year, and the weird thing is that most of them they just loved Iceland so much that they are going back every year to work there.
But yes, it is true.. you cannot bring anything with you that you used with horses out of Iceland. The tour guide told me that she had to wash her boots, gloves and riding-clothes multiple times when she arrived at the farm.
Btw, in the same town we also went whale watching, absolutely something to do.
Personally, we rented a mini van and we slept in camp sites...way cheaper and I just loved it. And no issues at all with English speakers everywhere, not even in small supermarkets on the way. Of course I am sure not EVERYONE speaks it perfectly but we never had communications issues with locals :)
If you manage to go, you will not regret it. I am sure about it :)