Connect app works when installed directly, using private token gets me 401

I’ve been testing my app by installing it directly from the descriptor and everything was working smoothly. Now, I submitted it to the marketplace and afterwards created some private tokens to continue testing (the app is not yet approved as of writing).

When installing with the private token, my app stops working because all calls result in 401

The request has not been applied because it lacks valid authentication credentials for the target resource.

Is this normal because the app is under review by the marketplace team? If not, what could be going on here?