you need to adjust your rear brakes properly.
the self adjuster mechanism frequently sticks, and then stops adjusting.
Take off the wheel, then pull the drum off.
You'll see a star wheel that has some threads on it.
Adjust the shoes until there is a light drag on the drum when you turn it by hand.
It is possible to do this from the brake backinf plate side, but it's a pita to do it one click at a time with a brake spoon.
If the brakes are adjusted properly then you can adjust the e-brake handle. I'm not sure where it's at on an XJ, but you should find it on the underside of the body, it will be a singel cable with an adjuster nut that "y"s off into the two cables that go to the drums. I'd be more inclined to believe that the brakes need adjusting though.
