Yes, a tucked in polo and a belt. A turtleneck is also fine in the cold, or a polo with a v-neck over top. Sleeveless is a no-no. And try to keep things tucked in with a belt, or add a vest (are vests "in" anymore??).
Also try to stay with more neutral colors until you know your trainer more. I ride in leopard print gloves and feel pretty badass with my coach because she is quite conservative. But no lime green breeches! And only ever white boots/wraps.
They say money doesn't buy happiness -- well happiness doesn't buy horses!