Got the lower shaft out today and looked, the bushing seems intact. Also the u-joints on it are good and I am unable to make it do anything weird with hand pressure
However I did notice some play in it when I was jerking it around in the vehicle so I went ahead and added a 1/4" rollpin per the idea by ricko70 in
this thread.
Reinstalled it and steering is slightly stiffer but the slop and pop is still there. Upon further inspection I think the popping problem is with the column, since I was able to wiggle the shaft coming through the firewall, and was also able to wiggle the steering wheel with the lower shaft disconnected--all told the column shaft seems to be loose in its housing. I also noticed that the input on the steering box isn't loose or anything but it has some play before the Pitman arm moves, maybe 1/4 of a turn. This is probably the source of the wandering, but I think the pop is in the column. I also made a leak in the rear seal on the steering box during removal so I get to make that upgrade sooner than expected.
Any more insights on this?