If you mean an English saddle, then to a tiny degree but not enough that it will possibly make a difference. Keep in mind that any horse is always gaining or loosing weight and muscle so the physical makeup of a horse is dynamic. The only time you need to have a different saddle for a second horse is when the structure of the second horse is different and the saddle is too wide (or narrow) for that horse.
Iridehorses is right...in fact...I know a couple people that will ONLY ride in THEIR saddle. One guy has a western saddle, and if his saddle won't fit the horse...he won't ride it (strange right?)... I know of two others that only ride in their saddles...but their saddles are wintecs and they use the changeable gullet system like a religion.
You wouldn't "wreck" the saddle, but you would need to make sure it fit both horses, do you don't "wreck" the other horse . My two horses have very different back shapes and cannot use the same saddles, even with creative padding.