Action Required - Atlassian Connect installation lifecycle security improvements

Announcements
, , , ,
163

Hi,

I’ve read this many times and it was never clear to me what happens with the shared secret from the install payload. The page doesn’t mention that each install event that’s signed with RS256 provides a new shared secret. Could someone from Atlassian confirm this behaviour? I had already confirmed it with the connect inspector https://connect-inspector.services.atlassian.com/ but I’d like to know if this is intended or a bug in the implementation of this new asymmetrically signed request feature.

The reason I’m asking is that we run a multi-tenanted setup in which caching is done for tenants. As it stands now the caching is preventing us to roll out the feature for all our customers because their requests to Atlassian won’t be valid due to the new secret they get when an automatic installation happens as a result of the descriptor change.

Moreover, if this is the intended behaviour, could you please update the post to mention this breaking change?

164

Hi, I am not relying on automatic registration in development mode, i.e. using:

AC_OPTS=no-reg node -r esm app.js

In newest ACE (7.4.0, 7.4.7), when you try to install the app manually you will get:

Authentication verification error (401):  JWT claim did not contain the correct audience (aud) claim

In previous ACE version (6.6.0) it worked well and all you had to do was to set baseUrl in descriptor object to ngrok url:

import ace from 'atlassian-connect-express';

const addon = ace(app, {
    config: {
        descriptorTransformer(descriptor, config) {
            if (config.environment() === 'development') {
                // this is to avoid conflicts between local and heroku installations
                descriptor.key = ADDON_KEY_LOCAL;
                disableLicense(descriptor);
                if (process.env.AC_OPTS === 'no-reg') setBaseUrlToNgrok(descriptor);
            }
           
            return descriptor;
        },
    },
}, logger);

function setBaseUrlToNgrok(descriptor) {
    axios.get('http://127.0.0.1:4040/api/tunnels')
        .then(response => {
            const httpsTunnel = response.data.tunnels.find(tunnel => tunnel.proto === 'https');
            descriptor.baseUrl = httpsTunnel.public_url;
            console.log('Running on ', descriptor.baseUrl);
        });
}

Is anyone using similar approach and knows how to fix it?

EDIT: I have hardcoded “baseUrl”: “https://xyz.ngrok.io” to make sure it is not a race condition and it still doesn’t work. When I set “signed-install”: false, everything starts to work again

165

I have figured it out. Latest addition of allowedBaseUrls basically a little bit broke ACE.

Now, the aud claim is always checked against allowedBaseUrls array, which is constructed as a concatenation of localBaseUrl property from config and allowedBaseUrls property.
This breaks ACE, because now, baseUrl from descriptor is ignored completely, meaning you can’t have static baseUrl :upside_down_face:
Also descriptor transformer is broken because of that.

Workaround: You can hack it by setting:

process.env.AC_LOCAL_BASE_URL = descriptor.baseUrl;
2 Likes
166

What is a

config.json

Our App is made with PHP back-end and JavaScript front-end.

I don’t understand everything of this.

Can I just left these like below:

"apiMigrations": {
		"gdpr": false,
		"signed-install": false
	 },
1 Like
167

Hi Maciej,

May I ask which version of ACE you are using? The issue you mentioned sounds very serious.
We enabled the signed install but did not notice the issue, so I am quite curious.

168

Hi @dciupureanu
We do not encourage apps to expect a single shared secret, and we have plans for returning back to per-installations based shared secret.

However, this change has not been rolled out to all tenants yet, but is enabled on Ecosystem Beta Group and Early Adopter tenants for testing. Also, this feature is only enabled for the apps with signed-install feature opted-in.

Just to be clear with the existing behaviour on all other production tenants, if you are installing your app manually from developer mode with your descriptor URL, you will always get a new sharedSecret. And if the install was triggered from the marketplace, your app will receive a single shared secret for all tenants.

Additionally, if your app had received a mismatching shared secrets from production environment, this is due a recent Marketplace failure that made connect to safely fallback by sharing a locally generated shared secret. If I remember correctly, there was an incident early this month that may have lead to this fallback mechanism.

169

Hi @matti.kiviharju
For customised connect apps

  1. Check if your app uses an authentication descriptor setting other than none. (There is nothing to do if your app is using none)

  2. Opt-in to “signed-install” feature from your app descriptor to start receiving asymmetric JWT token from install/uninstall hooks.

  3. Fetch the public key from our key sever using the 'kid' value extracted from the JWT header. (https://connect-install-keys.atlassian.com/\${kid})

  4. Update install/uninstall callback authentication to verify the JWT using the public key and with RS256 algorithm.

  5. Validate if the “aud” claim contains the correct app baseUrl.

170

Thanks for reporting this.
I’ve just patched a new version 7.4.8. ACE will now include the baseUrl which is modified via descriptorTransformer function, during audience claim check.

1 Like
171

Hi @chhantyal, I checked 7.4.0, 7.4.7.
@HanjooSong thank you for the fix, I will check it soon :slight_smile:

172

Dear @HanjooSong,

we switched our app (based on a homebrewn framework) and everything works fine. We closely monitor the logs and recognized that quite often, we receive install requests for instances at https://free-app-blitz-carbon*.jira-dev.com/wiki that reference a non-existing public key - the key store server returns 404. Is this some kind of planned testing or is something broken somewhere?

Best regards
Steffen

Edit: just asking because this messes with our “Did the change deploy successfully” metrics. If this is an expected behavior, we would ignore those errors for our reporting.

173

Hi @SteffenMueller
jira-dev.com domain is for internal testing environment. I believe some team has automated test running against that instances with your app included. Could you please post your app key so that I can trace down the owner of the instance and check if they can remove your app from testing?

174

Hi,

Thank your clarifying this, but unfortunately I still have some questions so let me see if I got it:

  1. If I opt-in I will get a new shared secret with every install in Production i.e. installing via MPAC
  2. If I opt-out I will not get a new shared secret with every install in Production i.e. installing via MPAC
  3. If I opt-out I will get a new shared secret with every install in a non-Production environment i.e. installing the addon manually in developer mode
  4. If I opt-out the behaviour stays the same in Production i.e. installing via MPAC, as it currently is without the flag being set at all

Sorry for this back-n-forth, but I find this thread and the communication about the changes this feature bings in all over the place and, sorry to say, in the typical Atlassian fashion.

Could you please update the documentation in this announcement, in the one about secrets changing and on the developer docs to reflect what each state of this flag means please?

Thanks.

175

@HanjooSong now that we are approaching the deadline of this change, can you please share with us if you have any analytics regarding how many active Atlassian Marketplace apps are still not compatible and whether Atlassian is considering a minimal threshold required attached to a final GO/NOGO decision on Friday?

2 Likes
176

Hi @remie
We have actually scheduled another email notification to inform you of our next stage, but looks like it may be sent on 1st of Nov at the earliest.
In short, we have decided to use AMS ticket for all apps that has not opt-in for this new authentication because we are still seeing a high number of apps that did not take any actions. We will not be enforcing any installations to break after the end of this week, but moving forward, the new remediation due date and the enforcement will follow the AMS enforcement policy.

1 Like
177

Hi @dciupureanu I understand your pain and I agree with you that including developer documentation in the announcement was not a good idea. (Unfortunately the announcement cannot be updated after 30days)

  1. If I opt-in I will get a new shared secret with every install in Production i.e. installing via MPAC
    → Yes, per-install based shared secret change will be followed within couple of months after you opt-in.
  2. If I opt-out I will not get a new shared secret with every install in Production i.e. installing via MPAC
    → Yes, it will maintain the existing behaviour. But you may still get a different sharedSecret if MPAC service or any other underlying internal Atlassian service experiences issue that relates to the shared secret. Also, when we need to rotate your app’s secret, it will take up to 24 hours for all Jira instances to be updated with the new one.
  3. If I opt-out I will get a new shared secret with every install in a non-Production environment i.e. installing the addon manually in developer mode
    → Yes, regardless of the signed-install feature you will always get a new shared secret for every install when you are using using “developer mode” to install an app. This is an existing behaviour.
  4. If I opt-out the behaviour stays the same in Production i.e. installing via MPAC, as it currently is without the flag being set at all
    → Yes, but the AMS ticket will be created for your app, and further actions will be tracked from there(https://developer.atlassian.com/platform/marketplace/vulnerability-tracking-more-info/)
1 Like
178

Hi @HanjooSong,

We are using AC Spring Boot. We enabled the signed-install feature with the necessary changes and released the new version to the marketplace. We are now getting the following message in the logs. Is it normal for install requests to fall back?

c.s.i.a.a.AsymmetricAuthenticationFilter : Failed to authenticate request, falling back to symmetric JWT auth
179

What will the SLA be for implementing this change?

1 Like
180

@HanjooSong please can you clarify? Will backwards compatibility be needed for both installed and uninstalled webhooks?

181

We cannot implement this until Oct 29, can you please extend the deadline! We were not yet fully aware how this impacts us.

182

No worries, it’s not going to be enforced yet!

1 Like