My gelding has degeneration in his stifles. You can have them diagnosed properly and they will give you the degree of degeneration if you really want to know, we had this done for him just to see how bad they were and what kind of stressors would be alright and what wouldn't.
Definitely is a muscle thing - once you get him muscled up you won't notice them much if at all (just general stiffness if he's had a couple days off). Horses with stifle problems never do well with lots of time off or just weekend riders, they need to keep good muscle and fitness to help suspend the joint properly and keep everything supple.
Work on hills, and hacking is especially helpful as it forces him to use the right muscles. The slower you go with hill work, the harder they work, so don't let him race up/down hills, make him go slow and use his hind end. Just work at the walk - no need to trot until he's really muscled up and then you can add in trot work. But walking really works the muscles on hills.
If it's possible, consult with your vet/farrier together - our guy did much better after having degree wedge pads put on his hind feet to increase the angles of his hind legs and relieve some pressure on the stifle. We were having his actually locking though, so this might not be right for your guy. Once the degree pads were on the locking stopped.
I also found that a good free lunge (no lunge line) in a round pen or an indoor arena before riding helped get him loosened up and warmed up before getting on. If they're really stiff it lets them work out the kinks on their own before you get on and ask them to work. Takes the stress out.