i say do it. if it bothers you, re-gear.
3.07, you probably have a MT? if so, it will be driveable, but non ideal.
i cant justify the smaller tires, 31's are a bare minimum tire for jeeps, imho. down the road, pickup 4.10's out of a small engine cherokee.