Brake Light switch Replacement

no rear brake lights on my '90 XJ and the trouble appears to be the brake light switch on the brake pedal. So I have the push rod bolt that attaches the brake pedal switch to the brake pedal removed but I cannot seem to disconnect the switch from the rod that goes through the firewall to the barke system. It has a round eye on the end that the push rod bolt also goes through. The FSM shows a sleeve that is in the switch does that need to be pushed out or this this a case of pull like mad until it pops off?

Any help is appreacited.