Believe it or not, I've found it depends on the car - in our Acura MDX, which has a rather steeply sloped windshield, the unit is closer to the air ducts and quite far from the driver - I often have to turn the blower way down or off to use the hands-free bluetooth feature. Conversely, in our Hyundai (more 'vertical' windshield than the MDX), the unit works fine with the fan on any setting below full-blast.
Either case, you gotta turn down the radio manually . . . a small price to pay (in NY) to comply with the hands-free law.