Did you put a bridle or a halter on your horse, J? Most, but not all, bridles have a bit (that goes through the horses mouth) and the reins are attached to rings on the outside. They tend to have several buckles (for adjustment) that join pieces together and a browband that goes in front of the horses ears and above the eyes. A halter (except a rope halter) tends to have rings that join the various pieces, no browband, and one snap hook on the strap closest to the horses neck. However, there are many, many different variations of both halters and bridles, so if unsure, post a pic and we'll tell you.
I would recommend you start with a halter, which as mentioned above, you attach the lead rope to the ring under the chin. The horses in CountryWoman's avatar have halters on.
If you are new to just halter and bridles, nevermind saddles, I can't emphasize enough that you should not be putting any kind of tack on the horse without some supervision. It's easy enough, but if done incorrectly can cause great problems down the line.
Likewise for leading. Since you are green to the lead rope, have someone show you how to hold the lead (as described above) and how to walk with the horse, how to tell him to move to the right, left, backwards and forwards. Once you have that under your belt, there are umpteen things that people could show you do continue on with ground work to gain confidence and get to know your horse. Sounds like riding is a long ways off, but you will have many hours of enjoyment with your horse in the interim. Riding is only one facet of horse ownership. Most of us enjoy all of the usual chores of horse care.
Tough question: is it still a chore if we enjoy doing it?