I have 5 XJ's, 90, 94, 96, 96, 00. They all are the same, slip something into the hole while the arm is pulled away from the window and then release, this allows you to pull the arm straight off of the drive hub.
Maybe someone put a front arm onto the rear on yours, dunno?