When motor is under load, vacuum decreases. No vacuum default is defrost vents. Yes, you have a vacuum leak.
When blower fan is blowing hard, and not much air comes out (anywhere), you probably have evaporator freeze-up. Usually there is not much that can be changed to fix this. As far as I know, XJ's do not have a evap temp sensor, although the high pressure sensor is supposed to help avoid the freeze-up. Freon charge can also be manipulated to try to help this.
The main things you can do to minimize freeze-up are: Keep windows closed to not keep letting humid air in and run in MAX or recirc mode for same reason. The evap has to get the moisture out of the air, which is what freezes up on it. Keep the fan on a fairly high level to promote air flow over the evap instead of leaving it on very cold and turning the fan on low. Turn the temp warmer instead of the fan lower. Hope this stuff helps you.