You also need to request the ACCESS_EMAIL_ADDRESSES scope as described in this post.
ACCESS_EMAIL_ADDRESSES