I'm not sure about later models, but earlier XJ's without AC have an extra idler pulley that mounts in place of the compressor. That pulley is rather hard to find, and expensive to buy. Unless your compressor clutch bearing itself is bad, and whining when the AC is off, the easiest solution is to leave the compressor in and let it be the idler. It does no harm in there, except for a little extra weight. This is the best solution if you contemplate ever selling the Jeep, since others might want to have the AC all there, even if it needs repair. Or, of course, you could yank out the plumbing and the condenser if it's empty anyway, and use the compressor for onboard air.
If there is still refrigerant in it and it whines when it runs on AC, and it bothers you when you run the defrost, just yank off one of the wires on the pressure switch that's mounted on your receiver-drier. It will think the system is empty, and the compressor won't run.
This is, of course, assuming that you don't care to fix it. It might be worth looking into that option first, if it turns out just to need a can of R-134 or some oil.