I’m no expert but saw your post and do have this working. Take a look at:
Specifically, I think you need to use the V1 endpoint added specifically for downloading the content using OAuth (no V2 version that I can see):
https://developer.atlassian.com/cloud/confluence/rest/v1/api-group-content---attachments/#api-wiki-rest-api-content-id-child-attachment-attachmentid-download-get
Apologies if I’m misinterpreting your question!
Mark