Don't pry on the nut.
Get two bits of flat steel stock (about 1" wide, and a little less than 1/4" thick, if you can swing it,) and insert them between the switch body and transmission case with the nut FULLY REMOVED. Also (and before you do that!) dribble some air-tool oil or ATF on the shaft, and go for a sandwich.
About a half-hour after applying the oil (OK, a big sandwich. And maybe a beer,) put the flat stock between the switch body and the transmission case, on either side of the shift shaft. "Shake and wiggle" the flat stock alternately, until the switch itself comes loose.
If you have difficulty, it may help to tap the switch lightly with a bit of wood.
When you reinstall, brush never-seez ON THE SHIFT SHAFT ONLY, and make sure the clamp nut threads are clean. This will help you take it off the next time.
5-90