Advance Notice: Connect Data Residency Customer Rollout

Hey @lexek-92,

Appreciate your concern regarding realm persistence for re-installations (i.e. re-pinning on reinstall) and understand the implications it can have when a previous installation uninstalls/reinstalls an app. As @SushantBista has previously shared, this is something which we are exploring and are seeking to understand better.

If any active participants of data residency have been experiencing challenges due to this, I would be open (and keen) to discussing and exploring the impacts which it has had. @lexek-92 I’ll reach out to discuss further.

1 Like

Hey @marton.kelemen,

Thanks for the question, just want to clarify my understanding correctly. In this circumstance you are already hosting the data in a specific region which is not one the same one which Connect is aware of (potentially before enabling data residency)? In this circumstance, you’d want to ensure that Connect is aware of which region that data is already hosted in, so that it could be represented as pinned and eliminate the need for a customer to have to request a migration?

With regards to the lifecycle events, the initial /schedule would inform you of the target region. Once the migration starts and customer data has been migrated (or in this circumstance, exists) within that region, you would respond to the /status request with ready-to-commit.

Where do vendors have to indicate that an app only stores data in the host product (region).

  1. Security questionnaire (is this the only place so far?)
  2. Connect app descriptor?
  3. ??
1 Like

Hi @UlrichKuhnhardtIzym1,

I assume you look for this: Privacy & Security tab
Here you can provide the necessary information:
See also the respective API method: if I’m correct, the appTrustInformation -> dataAccessAndStorage -> isSameDataProcessedAndStored is the variable that tells about this.

2 Likes

Hey @SeanBourke,

Thank you for your prompt reply.
I can confirm, your understanding is correct.

I can also provide you with a very typical use case:

Our app registered the customer to the desired location (e.g. to the EU) based on Atlassian’s routing choice (i.e. based on geolocation), even though the instance was initially not pinned. Later on, if the instance gets pinned by the customer, we still have to do an official migration process for our app’s data, instead of just labeling the app’s data as already migrated.

So, this means that our app has to go through the standard migration flow, even if there is no real need for moving any data? Of course, we can shortcut at least the heavy part, as you wrote, but…
I still have concerns regarding this.

First of all, what is not clear to me is that how much system downtime can such a “fake” migration induce. If I’m right, the host product must be shut down/“freezed” during the data migration process. Are there any other potential side-effects, too? (provided that otherwise our app would be the only one participating this data migration session, these effort are all in vain)

There is another, not that much of a technical issue, but still an important aspect: how can customers get informed about the opportunity/necessity of a data migration at all? I think, following this concept they have to trigger all the events and then see the outcome, therefore, if they do not take the initiative, they won’t be aware of the fact, where their data are stored. Of course, if it’s not crucial to them, then we also can live with it, but a bit more proactive attitude could help a lot…

Apart from these, I could imagine that by adding an extra data storage location field to the migration overview (for which, the response could be obtained by another hook preliminary to the migration => similar to the /schedule hook), the admins could assess their apps’ statuses better, without the necessity to initiate an entire migration in order to get the answers for such questions.
As an alternative, an app could indicate in the response given to the /schedule event that the data are already migrated to the specified location, so subsequent hooks shall not be called. However, this could only tell the user, whether their data are already in a requested/target location, so the first approach seems to be more generic and user-friendly for me.

Anyway, if the implementation of such an assessment step would require too much effort, then we’ll go for the suggested solution. It can surely work, even if it’s suboptimal in some scenarios.

Best,
Marton

1 Like

Hey @marton.kelemen,

Thank you for the well thought out reply. My apologies for the delayed response, I wanted to ensure that we address each of the questions you’ve asked - hopefully the below helps.

If your app supports a regionBaseUrl of EU and Connect directs traffics for a customer’s app installation to EU based upon their proximity to that realm, then they will be deemed as being pinned to the EU realm.

  • If the customer’s product exists in (or later moved to EU), then that customer would see the app as Pinned to the EU region.
  • If the customer moves their product to a different realm which you also support in your Connect descriptor, then they will see your app as Eligible to move with a current location of EU
  • If the customer moves their product to a different realm which you do not support in your Connect descriptor, then they will see your app as Not Eligible to move with a current location of EU
  • If the customer had installed the app prior to any data residency realms being supported, then they would likely be directed to your baseURL and will appear as Eligible given they will be deemed as being in Global (or Not Set).

The /status endpoint is queried on a 15 minute interval, meaning that in this circumstance, the maximum duration of product downtime would be 15 minutes. It’s worth noting that the scheduled migration period can include multiple apps and the product downtime is impacted by the longest running migration task; so if a different app takes longer to migrate in the window, then the product will be unavailable until that app also completes its migration. I’d be keen to understand if you feel that for live migrations (where you have to move data between regions), if a 15 minute polling period is sufficiently frequent?

At the moment, this is provided through the via the Admin UI to provide admins visibility of where app data is currently located. A previously proposed improvement to have prior awareness (i.e. inverse app awareness) of a mismatch with a customer’s realm based upon the feasibility of a tenant pinning API would also provide the capability to support customers in understanding when an opportunity exists to realign data. There is no available scope or timeline available for this at the moment, however.

Appreciate this feedback and understand how it could create efficiencies in the process. It would be great to understand the frequency you have to perform fake migrations once this is live, as this would inform the potential for an improved customer experience. As noted earlier, the minimum downtime for a customer’s product is 15 minutes (pending all app migrations are completed in that window).

1 Like

^ can I please get an answer on this one?

Hey @nathanwaters,

If you only store data in confluence and Jira, then you would not need to declare the regionBaseURLs in the descriptor.

Appreciate your concern regarding the signalling of apps in the Not Eligible category and have shared this feedback with the Admin UI team. Customers will be guided to review app’s Privacy and Security tabs to better inform them about what data an app utilises and how it stores data.

2 Likes

Thanks. I think apps that only store data in Confluence/Jira should be under their own category.

If it’s just a link to the security tab on the marketplace, customers are 100% not going to bother going through those steps… they’ll just incorrectly assume the app is not handling regional data.

So you’re punishing apps who have gone to extreme efforts to keep all data inside Atlassian’s stupid 32kb content/entity property limits.

If you’re going to place such apps under the “Not Eligible” category then it makes perfect sense for me (and every other vendor) to add fake regionBaseUrls matching the baseUrl so the reality of how data is handled gets properly conveyed to customers.

Many of the largest apps on the marketplace already do this: regionBaseUrls are identical to the baseUrl in their descriptor. Very easy to find examples: start with the apps in the screenshots at the top of this thread.

Is this the suggested approach to correctly signal so my apps don’t get thrown into the “Not Eligible” category?

4 Likes

I totally agree with Nathan. Please make another category for apps that store data in Confluence and Jira. In fact, that category should be prioritized.

3 Likes

Also agree with @nathanwaters ,
Apps that store data in Jira/Confluence can offer data residency, in contrast to Forge apps. That should be highlighted for customers.

2 Likes

@nathanwaters , @chhantyal and @marc thank you all for your feedback. I’ve shared this with the customer facing UI team and we are exploring options to better represent an app’s status within the Admin UI. While this will not be available within the initial beta release to customers, we will provide more details as this progresses. In the interim, I recommend ensuring that you have accurately completed your Privacy and Security tab responses, as customers will be guided to review these to better understand how apps manage their data.

1 Like

Hello from Stiltsoft Europe!

We also would like to contribute to the subject.

As one of our add-ons Table Filter and Charts for Confluence doesn’t access, store, or otherwise process any personal data since all the page content is stored within the Confluence page, the option for it is to be Not Eligible.

However, our customers are deeply concerned about data transition, which in fact takes place through our servers on occasions of content exporting and for some of the macros parsing. These cases are directly related to customer data handling and as such are part of the Data Residency.

In light of that, we believe it would be better actually not to consider such apps as ours Not Eligible but instead add some kind of indication/flags in the app descriptor pointing at Data Residency actually supported, with the data stored only in Confluence.

Hopefully, this matter will be accounted for accordingly

Many thanks!

5 Likes

:wave: @NikitaKamai and team,

I want to clarify that there are cases where the app will be presented as Pinned:

  • a customer site is located in US based on their origin of access when the site was created → the customer installed the app after the US location was supported in the descriptor via the regionBaseUrls tag → the customer then moved (or moves) the product to the US location
  • a product has been moved to the US location → the customer installed the app after the US location was supported in the descriptor via the regionBaseUrls tag

The same applies to the other supported regions by the app.
This is because the app is currently supporting data residency pinning based on the descriptor via the regionBaseUrls tag.

If, instead, a customer moves the product to a different location from the one where the site was created, the app will only be shown as Eligible once it supports data residency migrations.

In the case of this app, once data residency migration is supported, the Not Eligible option will only be shown for those regions that are not supported in the descriptors.

If your plan is to support all regions and the migration option, Not Eligible will not be one of the possible options for this app.

As Sean shared above, we are looking into options for a better representation of the app’s status. In the meantime, please provide the answers to the Privacy & Security questions for customers to review the data residency offering of the app.

Hope this helps,
Caterina

1 Like

The gist here is that our app does not store any customer data, as it’s actually stored by Confluence as part of its page, for example. Therefore, Table Filter and Charts add-on requires no data residency migration in fact. While so far it looks as now the status for the app is to be Not Eligible because of this, which in reality has nothing to do with how things are.

We will greatly appreciate if your team could consider this circumstance accordingly.

Thank you kindly,
Nikita

1 Like

Hi @NikitaKamai,

I appreciate that similar feedback has been shared by the community and this has been shared with the customer facing UI team. We are currently exploring how we can better represent app statuses to customers. In the interim, I recommend ensuring that you have accurately completed your Privacy and Security tab responses, as this is likely to be used by customers to better understand how an app utilises and stores their data.

While adoption of data residency and realm migrations has primarily been focused around in-scope data, I also appreciate that some partners have adopted data residency within their Connect apps for other reasons, such as server proximity to customers. If you already support data residency, or believe it would be valuable, extending that to also support realm migrations helps to ensure that your app pinning is aligned to the customer’s product.

1 Like

Hello @SeanBourke,

The point is such migration will be fake in fact, as there will be nothing to migrate for the reason we have expressed initially - the app does not store customer data in any form.

Regards,
Nikita

A quick update regarding the representation of Not Eligible apps within the Admin UI. While we continue to explore alternatives, in the interim we have added more verbose explanations to admins as to why an app may appear within this tab.

Admins will now see a panel within the page which notifies them that there a multiple reasons as to why an app may be appearing within Not Eligible:

  • Some apps store data exclusively within the Atlassian product, therefore they do not have any in-scope data stored which needs pinned or moved
  • Some apps don’t support data residency for the product location.
  • Some apps don’t store any in-scope data. Hence, they need not be migrated or pinned to a location.
  • Some apps don’t support data residency migration between locations.

As per my previous message, I recommend ensuring that you have accurately completed your Privacy and Security tab responses, as customers will be guided to review these to better understand how apps manage their data.

2 Likes

Hi all,

We just announced the beginning of the customer EAP for the new app data residency UI. This new UI will allow customers to see apps’ data residency status, and schedule app data residency migrations. We appreciate your feedback and will use that (and any customer feedback we receive during this initial rollout) to improve how app data residency status is reflected in future iterations of the UI.

Now that the rollout has started and customers will start to have access to this feature, we want to provide the ability to see and use the UI directly. We’ve added additional information to the develop documentation with instructions on how to submit a request for the feature to be enabled in a development site.

Due to the nature of this feature being part of the https://admin.atlassian.com UI, the sites using the Developer Canary Program apps will not automatically receive this feature as part of the first release cohort.

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.