While I don't think that you are intermediate yet, ask your teacher if you can maybe alternate between the two levels until your confident enough to be competent in the intermediate class.
I don't mean this in an nasty way; but you don't like being held up by the beginners but I wouldn't want to hold up the better riders if I was ready yet either. You have to find that balance!
Is Pebbles your horse or a school horse? Maybe talk to see if you can give another horse a try.