If it's like other Android devices it uses the Notification sound which is a global sound used for all notifications, but in Jellybean they added the ability to disable the notification by app which should be nice.
To set this go into your Settings > Apps > side to All > and tap Gmail. Now uncheck "Show Notifications" . You won't get any notifications or sounds when a new email arrives, so that may be less than ideal -- but this will stop the sound from appearing.
One quicker way to set it is also to long press on the app in the Notification window, if a notification exists, and it'll bring you the settings page as well to disable the Notifications.
Hope this helps --