Doesn't sound like brake booster. Brake booster should not depend on actual vehicle motion, but usually will change when you apply brake.
See if you can narrow it down, both in location and type of sound. Does it seem to be cyclical, or change consistently with speed? If so, consider rotating parts such as axle u-joints. Does anything you do while driving alter the sound? A sound that changes when cornering, for example, points to something like an axle or wheel bearing that changes under load. If it seems more just like a steady whistle or whooshing or whining noise that is not related to engine speed, consider air leaks. Windshield air leaks can cause a variety of sounds depending on where the air hits. Other sounds could come from leaky HVAC controls, or a leaky vac. line in the cruise control.