The solution for confidence issues where jumping is concerned always seems to be lowering the jumps and do them until you are bored senseless with them and can't wait to raise them again. If a x-rail makes your nervous, go back to ground poles. There are plenty of exercises you can do with ground poles such as lengthening/shortening stride, incorporating circles over them, bending lines, related distances, etc, to keep your mind engaged and you and your horse working without overfacing yourself or boring your horse. Don't let anyone, including your instructor, push you past your comfort zone until you know you are ready for it. When you are ready, you will just know.
I have to say I disagree. I personally can think of many times where I felt nervous and had no confidence trying something new. (not only riding, but I am a skiier/snowboarder as well) What I ultimately gained from doing things I once feared was confidence and progression. If you have a good instructor, they should be well aware of what you and your horse can and can't handle.
I have been back in the saddle for a year now. I grew up riding dressage, and never jumping higher than 2'6, based on my instructors preference. I finally, after 6 or so years out of the saddle, had the means to afford riding again and found a hunter/jumper trainer who had wonderful credentials. This summer she had worked me up to competing in 2'9 hunter classes. One show, we were in the warm up ring when all of a sudden at fence #4 my horse refused, and I fell off. He refused every jump thereafter several times. Having minimal jumping experience, and riding horses who like to jump, I felt incredibly overwhelmed and that I was over my head. After all, I had never had a horse refuse at a jump. We ended up finding a rock in the cleft of his frog, which was obviously causing him pain when he landed. Anyway, rock was removed, but both the horse and I had developed a fear/bad association with jumping, and the refusals continued for weeks following at home. My trainer pushed us both (having me correct the issue) and has gotten us to where we both know what our job is while jumping, and if he thinks about refusing I know how to handle the problem appropriately. However, If my trainer didn't push me I would probably still feel afraid, and would have not had that experience to progress as a rider.
I personally believe that at some point or another, every rider will have some sort of fear, which is natural due to the nature of our sport. However, you can only progress as a rider with every new situation that takes place.