Electrical Charging Problems?

My XJ is having some problems charging as of late. Its the 4.0 and I've been pulling a code 41(Charging Relay Switch or something like that) but it hasn't been acting wierd til recently. Last night when I got off work it wasn't charging very well, I turned my CD player on and my lights and it died. I could drive with the headlights on but they were EXTREMELY dim. I took out the alternator and got it tested and it came back fine.

I also noticed as of the past two days on my drive to work it would be charging at like 9ish, then randomly jump up to almost 14 for the rest of the trip. As I was driving around it seemed like it would hold 12 as long as I didn't have anything on(lights, cd player, etc).

Also, when I was putting the alternator back on, one of the small pins that the black box goes over broke off. Are those important or do they just hold it down? I haven't noticed it acting any different with out it(the little poles that have a 8-9mm nut on them and are really close together on the back of the alternator).

Any ideas for me?