The first time I got on my qh filly was in the water. I start my horses bareback for the first few rides anyway. We have a large pond that we take the horses swimming in regularly. I had a bathingsuit on and this filly happens to love water, she started swimming laps as a yearling just for fun. She would always follow us in with the other horses anyway, so one day I led her in and swam up on her back. She and I had our first real ride in about 4' of water.
I wouldn't recommend this for everyone, I did it with a filly that I'd had most of her life, she had a sound mind, had done quite a bit of ground work, and already loved the water and had a great amount of trust for me. I personally wouldn't want to do it with a horse that was already scared out of its mind, but how I train I wouldn't get on the "out of its mind" horse anyway until the mind was restored.
Some advantages - the water gives that "float factor", so your weight being up there and moving isn't going to have the same unsettling affect on the horses balance. If you do it bareback and there isn't anything to get caught on, falling really doesn't hurt. Not to mention, here, where we have summer 3 months out of the year, its nice to get out of sticky jeans and breaches and get some sun on my white legs while enjoying the company of an inquisitive filly. "Bug" and I still go swimming all the time, I've even used her extreme love of water to teach other horses that have a fear of water to accept and enjoy it.
I think ultimately, it depends on the horse, your personal training style, and whether or not you are a good swimmer. :)