In addition to the good advice above I will add that the XJ before '97 has an additional weak spot. The forward end of each brake cable sheath is anchored at a bracket that is spot welded to the floor. After a long time, these brackets will flex and the floor can crack around them. Once they're weakened, all attempts to get good rear holding will be frustrated, because when you pull the brake, the cable will always try to straighten against the sheath before it pulls on the brake. Even if the floor is not cracked, you should look at the brackets closely and see if there's any flex or if the spot welds are starting to pull.
I fixed this on my 95 by drilling through the brackets and bolting the brackets firmly to the floor, with a reinforcing plate on the top. It made a huge difference.
This problem may be more prevalent on stick shifts, since their hand brakes get more use, but it's worth checking. 97+ models have a different anchor point which appears to be much better designed.