Typical of a vacuum leak...
I think its very possible that you have a vacuum leak. When you unplug and cap off the vacuum line to the regulator your taking a small load off of the system possibly allowing the system to run as it should. Plug in the vacuum line, and spray carb cleaner (while the engine is running of course) directly on each section of vacuum line that you see. If the idle changes at all (hesitation etc included) then you have found a vacuum leak. Fix said leak and continue to make sure you have no more. Then you can determine if its actually the regulator or simply a vacuum leak. Im betting is the latter...
Dont forget that a sticky IAC can do the same things. Make sure you clean that puppy up also... In this instance however, since the problem is fixed when fooling with something vacuum related, that to me says hey... the problem is probably a vacuum related problem!