Just before the CPS went out in the wifes 96, the idle got pretty bad and the ignition would break up some at higher RPM's.
Vacuum leaks can mess with your idle. The vacuum taps on the intake manifold are on one end or the other mostly, so you can end up with lean and rich cylinders. If the vacuum leak is small, the IAC will compensate some, so the idle isn't excessively high. If it's a large vacuum leak the idle will often be high. Vacuum leaks rarley cause very noticable problems at higher RPM's.
Uneven compression will also cause idle problems, unburnt fuel and/or excessive carbons can mess with your O2 sensors.
O2 sensor can mess with your idle and hurt your top end. One could be flacky enough to cause problems, but not bad enough to throw a code.
Low fuel pressure will also cause the problems you discribed. As will a partially plugged Cat. I had to replace the Cat on three XJ's an 87, 88 and a 96 all in the 130-160,00 mile range.