Using marketplace private token results in 401, but not with local descriptor

SOLVED: see below.

I have an app pending review to the marketplace. The shared secret generated after installing via private tokens from the app’s marketplace config page do not work to make calls to the various Confluence endpoints, although the app installs just fine. If I try to use the app after installing it from a local descriptor with a different clientKey, everything works fine.

I have narrowed it down to the actual token being the problem, but that still takes me nowhere. Here’s what I did:

  1. I launch my app on my laptop and start ngrok.
  2. I go to “Manage apps” in my Confluence cloud instance and click “upload app” and enter the url to my ngrok json descriptor.
  3. The installation works. I then connect Slack and everything works with my plugin (it’s a Slack integration).

Now with the private token, with my app running in Heroku:

  1. I go to the Log in with Atlassian account/private-listings and create a private token (hiding the id here in this url because I don’t know if that’s sensitive)
  2. I go back to “Manage apps" in my Confluence cloud instance and in “upload app” I enter the link for the latest version of my plugin, which I got from the previous step (1.0.1-AC)
  3. The installation works. Then I connect to Slack via the plugin’s “Get started page” and still everything is working.
  4. Now, if I try to run app in Slack, I see the first modal pop-up (which means the app is running fine), but then it doesn’t do what it’s supposed to do (search Confluence with CQL and create a Confluence page). In the logs I see these calls failed with
HTTP Status 401 – Unauthorized

Type Status Report

Message Your presented credentials do not provide access to this resource.

Description The request has not been applied because it lacks valid authentication credentials for the target resource.

So this could still be some difference between my local environment and Heroku, so I manually swapped the shared secrets from the two scenarios and now the local instance in my laptop (using the shared secret originating from the private token) does not work, showing the exact same 401 error, while the shared secret originating from my ngrok now makes my Heroku instance work.

The code is exactly the same in both scenarios above. The app descriptor is also the same, except for the key which is different (added a -dev suffix to avoid collisions) and the base url which points to my ngrok instead of heroku. Otherwise it has read and write permissions, auth via jwt etc everything is the same.
The only thing that changed was the shared secret and how the app was installed.

I can’t think of anything else to test. Any tips are much appreciated.

SOLVED: I took a nap and approached this with a refreshed mind and I thought

  1. It’s unlikely this is a bug on Atlassian’s side, otherwise more people would complain.
  2. It must be something in the environment.
    Turns out it was a hard-coded variable in the way “iss” is calculated for JWT. I had to implement JWT calculation on my own, because my app is written in clojure and this slipped through during some testing.
    Leaving it here for future reference and learning that hard-coding can hurt you bad.