I'm 5'8, 145#, wear a size 8 pant, and found the mediums to be a little small. Maybe they will stretch?
If you have a flat butt you could probably get away with it, but I found the large to be more comfortable.
Both sizes come up higher on the waist, which I prefer for workouts.