Even if you have a small leak in the vacuum system there should be a check valve in the line supplying the climate controls. The purpose of the check valve is ensure the climate controls stay in the last position they were placed in during low vacuum situations.
This was found to be a problem on the '97s and early '98s and was resolved by installing the check valve. I had to install a check valve shortly after purchasing my XJ in Oct '97.
The check valve should be installed in the vacuum line near the refrigerant accumulator/drier before it goes thru the firewall.
At 70 mph my engine RPM is 2850 with 4.56s, 32" MTRs and AX-15.