I guess my feeling is that 50 - 75% of a stallions value is in his personality. If this horse can't be easily caught, and is slow to warm up than really those are signs that he would not be a good stud. There are a lot of pretty stallions who are difficult to work with. In my opinion those are to difficult work with to be of value as breeding stock. If he is not easy to handle at this age, when he realizes he is a stud and all that goes with the stud could make him harder to handle. If you do decide to keep him as a stallion than you have to put all the work into making him 100% great to handle. He can't be 95% or even 98% he needs to have 100% perfect ground behaviors 100% of the time. At this time, he does not have those and that needs to happen.
Housing is another concern. Where will he live? Can you safely keep him away from the mare and possibly (probably) the geldings as well?
The other concern is that he is sensitive to anesthesia and is that an indication of other less then ideal genetic conditions? I would just worry about him passing on that drug sensitivity to offspring. Likewise, if his behavior worsens and you have to sedate him for routine things that can complicate life. Ie. If he can't be easily handled and is not safe for many vet procedures, or farrier procedures. Which would add to his risk.
I would geld him, but I am going to say geld in 99.9% of the cases because stallions come with a whole lot of extra stuff that complicates life.