The Bluetooth stacks are not the best on a Nexus device, this is because Google won't pay licencing for proprietary software.
However, I use AKG BT Headphones and a JBL Charge with all my Nexus devices (Nexus 4, Nexus 7 2012 & Nexus 7 2013) with no problems...
This could be a compatibility issue, do you have access to an alternative BT device for testing purposes??
Sent using The Force