How regularly should you be exfoliating your face?

Is there such a thing as too much scrub?