Moving away from per-installation shared secrets for apps

confluence-cloud
announcement
jira-cloud

#1

Hi,
Currently, the App framework that backs Jira and Confluence sends a unique client key and client secret pair to your App upon each installation. Today, we are announcing that we intend to post the same shared secret for every installation of your App; this is part of an initiative to bring Jira and Confluence inline with the new Atlassian platform for developers.
Please read through the announcement blog post for more information.
Please feel free to provide feedback or ask questions here.
Regards,
Norman

Update: if you are interested to verify that your app works correctly please read below.

The sync process is very similar to an install-uninstall-install sequence that causes Connect to generate a new shared secret and sign it with the old shared secret for the second ‘installed’ lifecycle callback. By ignoring the uninstall event an app developer can simulate the shared secret sync operation that will happen during migration. Here is what you can do to verify your app will behave properly during migration:

  • Disable the uninstalled callback handler. One way is to modify the descriptor and map the ‘uninstalled’ lifecycle event to a non-existing path.
  • Start the app in the dev environment.
  • Install the app using its descriptor URL. Your app will receive an ‘installed’ callback.
  • Configure the app and make sure it works properly.
  • Take a note of the exiting shared secret in the app database.
  • Uninstall the app via add-on management. Remember that you are ignoring ‘uninstalled’ callbacks?
  • Install the app again using its descriptor URL. Your app will now receive another ‘installed’ callback, providing a new shared secret in a JWT token that is signed with the old one.
  • Make sure the app still works properly and the shared secret is changed in its database.

If you had any issues, please feel free to reach us via this thread or by creating a new service desk ticket. We have an internal process to dry run the migration on a test instance.


#2

#3

Hi team! Thank you for the blog.

Few questions

  1. How are you going to ensure that current hosts with their own shared secret will continue working? Would you send install webhooks with the same client secret to everyone?

  2. After the migration, Will it be possible to delete the shared secret value from every host as it is same for everyone? And then having a secure env property with the client secret?

  3. Would we need to refresh the access token generated from the client secret every time is expired? Supporting access token developers would need to implement if you are not using the supported libraries right?

  4. Is it safer to have one client secret for everyone? What if it gets compromised? Would this person have access to all the hosts?


#4

Hi,
Thanks for your interest. Here are some answers:

1- We will not change the underlying infrastructure, the new unique shared secret will work in the existing framework (one copy in Jira/Confluence database, one copy in the app database). We will send the single client secret to the same URL for each app, but once for every Jira/Confluence instance it is installed on. This will happen gradually. As an app developer you will notice the incoming requests over a few days/weeks.
2- This is exactly what will happen. We will update our platform and client libraries so that there is no need to store this information in database. There is only one secret to manage, which can be easily provided as an environment configuration.
3- If you create long-lived access tokens, they will need to be re-generated otherwise user operations will fail. Jira and Confluence generate new JWT tokens on every page load.
4- The same methods can be used to compromised single shared secret and installation-based secrets. If there is a way to compromise this single shared secret, there is a way to compromise all of installation-based secrets. All Jira/Confluence cloud instance use the same data-storage and same level of security in communicating with the apps, and we don’t have a weaker link (as you could find in Server). What this new model provides, however, is that we can much more easily revoke a compromised shared secret.

We are still investigating the security aspects of this project as we go, and we will review the whole thing again from security perspective once we have the first version of the solution.


#5

Hi,

I’d like to clarify the above comment.
As Norman have mentioned we are changing the way shared secrets are generated and communicated to apps. I that sense we see it as a two phase migration project. In the first phase we don’t change anything in the way apps currently work apart from using the same secret for all the instances. That does involved a secret rotation for installed apps that reuses existing app update flow.
After the second phase that it will be possible to use one secret and not store it in the DB. We are also working on a centralized platform for app developers that allows for app management. This will make it easy for developer to change shared secrets without any migrations. In fact it will eliminate the need for a secret to be sent in app installation payload further securing it. This model is consistent with existing industry best practices where a single unique key is used to identify the app itself, regardless of where it is installed.


#6

Please make it possible for app vendors to also trigger the exchange (i.e. creating a new) shared secret when implementing the


#7

For of us that are on ACE/SpringBoot and paranoid - is there anyway for us to verify that things work ahead of time? (Do you have a test utility?)

If for whatever reason the credential upgrade don’t work - what is the impact for an existing customer and how would they see this? (I want to prepare our support team).


#8

Fear not - we are definitely going to have this. It is crucial for us to enable vendors to self-service such operation with ease.


#9

That’s an awesome combo! We have manually tested a sample of addons to perform correctly. That included ACE/SpringBoot addons. But we will be more then happy organise another test with your addon if you’d like. Unfortunately we don’t have a test utility that we can share with you. But we can coordinate so you will be able to observe the effect.

It is really important for any addon to correctly perform credential upgrade as it is part of update flow. That is every time a new version of an addon released (a descriptor published to the Atlassian Marketplace) - this update flow is triggered. So if this flow is broken - addons won’t be able to auto-update. Or worse yet - won’t work after an update. That is for end-users the addon will appear broken as it will attempt to use JWTs signed with a shared secret that addon service won’t recognise.


#10

Thanks Daniel for your interest. I have updated my original message with a simple verification procedure.


#11

I made @natashbar simple verification procedure update a blog post to ensure that more developers see that bit of helpful information.

Please test your apps for readiness for the migration and let us know if you have any issues.


#12

Hi @rwhitbeck, I’m looking at your testing steps, and the second to last bullet point is this:

“your app will now receive another ‘installed’ callback, providing a new shared secret in the payload of the request, but with the request signed with a JWT based on the old shared secret.”

If the next time they decide to install the same plugin, we receive an install web hook that is signed with a previous shared secret, how would that work if we don’t have that shared secret anymore?

My assumption is that a user cancelled a plugin and uninstalled it, and we performed cleanup on our end to remove data about that particular instance from our side.


#13

If you don’t have the data it would act the same as if it was initially installing the app.


#14

Hi @natashbar @rwhitbeck,

What’s the timeline for this change to complete? In my case around 80% of the hosts are migrated to a single shared secret. However, rollout to the remaining 20% seems to have stopped. Is this expected?


#15

Hi Tobias,
Except a few add-ons that require some fixes, we have migrated Marketplace installation of all add-ons. However, we have not yet migrated the add-ons installed in UPM dev-mode (using descriptor URL).
If you are talking about Intercom for Jira, our records show that we have already migrated all production instances except 2 (both are dev-mode installations).
Please raise a service desk ticket (on https://ecosystem.atlassian.net/servicedesk/customer/portal/14) if you want us to look into this further.
Regards,
Norman


#16

Thanks @natashbar, I openend DEVHELP-1688


#17

Hi @tbinna,

If possible, we would be grateful for sharing the outcome of DEVHELP.

Some of our customers/hosts haven’t been migrated too. Atlassian says that the attempt to migrate them was unsuccessful for some reason and we are stuck at this point.

Hopefully, you will have more luck.

Cheers,
Jack


#18

Hey @jack, sure I will post an update here when we figured it out.


#19

Based on my understanding, there are cases where the customer has uninstalled the app, or their evaluation/subscription has expired. These customers will still be present in the database and it is hard for an add-on developer to find the current status of those customers without checking them out one by one.
When we migrated the credentials for Connect apps, we only did it for the active installations and ignored the disabled/uninstalled/expired ones. This will explain why we might see an app as completely migrated, but there are still a considerable number of records in an app’s database still having the old shared secret. We also haven’t migrated installations done in dev-mode via base URL. We are currently discussing this and fixing our strategy for migrating local installations.
The end goal is to move away from having to read shared secrets from database. When we get to that state, you can simply ignore/remove those records from your databases. However, in the meantime, if a customer re-installs an app they had previously uninstalled, it would work as usual and they will be moved to use the single shared secret as part of installation.
Stay tuned for more updates about the next phases of the migration.
Regards,
Norman