Three times now I had the same issues. I cleaned the IAC and it seemed to help some for awhile, the first few times it happened. The first few times it was just a sticky IAC, mostly on a cold motor. Another time it was my alternator failing and the third time it was a battery on the verge of a catastrophic failure.
Cause and effect, I replaced the alternator and my (periodic/sporadic) IAC problems disappeared. A year latter I replaced my battery and my IAC problems disappeared. There was definitely a connection between my lazy IAC and my charging system. I just don't know exactly what it was. Best guess is the IAC needs all the available voltage it can muster to work well and some sort of charging issue, that may be hardly noticeable on the volt meter or the way the starter turns over, may affect IAC function. The only noticeable indicator I had was my headlights seemed not as bright as they used to be.
With the IAC, if I gave it a little pedal the motor would run fine, my IAC tended to stick closed, which constipated the airflow at idle. When my CPS was failing, I had to coax some RPM's out of the motor by feathering the pedal. The IAC and CPS would act much the same (sometimes) when I was coming to a stop, at low RPM's the motor would stall (IAC) and/or miss (CPS) and the idle would fall.
Just a possibility.
I also had a MAP fail, the motor ran really rich and would idle very low (if at all). Clouds of soot came out of the exhaust.
Figuring out if it mostly happens on a cold motor or a hot motor may help with diagnostics. My IAC seemed to be worse on a cold motor, low battery after a few short stop and go trips before the battery had a chance to recharge. When my CPS failed it seemed to be worse right at the transition (140-160F) from a cold motor to operating temperature. It my have been something specific to my XJ, I also had a lazy O2 senor at the time.
Just some ideas.