Thank you all for reaching out with your questions about Data Residency. We recognize it’s been a while since we published any updates here in the Developer Community.
Last month, we published an update on our Developer Blog with an overview of our joint approach to evolving customer trust needs for cloud apps. This update included several upcoming changes to enhance transparency and control for cloud app customers across a number of trust topics, including data residency for apps.
Currently, Marketplace Partners can pin app data in Australia, Europe, and the USA for cloud apps built on Connect. This will allow for new installations of supporting apps to pin the app data to the same location as the host products. You can find details here.
What is the next major milestone for app data residency?
As we mentioned in the blog post and in the latest roadmap webinar, the next data residency milestone is realm migration support for Connect apps. This month, we are planning to release Connect realm migration service and APIs as an EAP for Marketplace Partners, to enable you to start integration and testing for apps to support realm migration. With these new APIs, Connect apps will be able to support migration of app data to match the parent product realm.
Please note, customers will not be able to schedule app migrations upon this upcoming EAP release. The EAP release is intended for Marketplace Partners and developers to start the integration and testing to support realm migration for your apps, prior to the customer experience release (which we are targeting for first half of 2023).
As part of this upcoming realm migration EAP release, we will update the developer documentation with updated list of supported realms, details of the new migration APIs and information on how to test app realm migration.
More details of the upcoming realm migration APIs is available in our earlier community update.
In addition to offering support for realm migration, we also plan to enable app data residency (realm pinning and migration) in Germany by the end of this year.
What is coming up on the customer-experience side for app data residency?
Following the Connect app realm migration EAP release for Marketplace Partners, we will also release features to help customers better understand and manage app data residency, including:
- A privacy & security tab on Marketplace listings (in the coming month we will provide more information on this feature and how to leverage it to promote your trust investments). This will help partners share app data residency information with customers while they’re searching for apps.
- A self-serve customer realm migration experience EAP on admin.atlassian.com to help customers view data residency options for their existing apps, and schedule their own app data migration windows.
We are actively working on both of these features right now, and expect them to arrive in the first half of 2023.
When will Forge support data residency?
Currently, Forge does not provide multi-region support. We’ve prioritized delivering data residency support for Connect apps first, to enable data residency for the largest number of Marketplace apps. However, we know that Forge support for data residency is a highly anticipated feature.
Data residency for Forge will enable multi-region support for the Storage API, providing data residency compliance to customers. As Atlassian will deliver and manage the relevant multi-region infrastructure and provisioning, it will make it easier for Forge-hosted apps to offer data residency.
We are working on plans to get data residency on Forge, and below are the factors affecting the timeline:
- We are currently in-progress to integrate with a new internal data store for Forge hosted storage. This new data platform will help us to deliver additional storage capabilities for Forge, including support for structured data and query-by-value (more details here). Data residency support for Forge-hosted storage will be our key focus area after we have finished the migration to this new data store.
- Multi-region support for Forge hosted storage may result in latency, due the geo-distributed architecture of the user, Forge’s hosted compute, and Forge’s hosted storage. To address this, we are planning to support multi-region compute for Forge to ensure low latency when Forge data residency is available. Multi-region compute is identified as a pre-requisite for Forge data residency, and we are still assessing the delivery timelines.
This is a complex body of work, and we want to be transparent about the considerations and dependencies. We will provide periodic updates as we make progress and lock-in timelines.
App data residency roadmap
As we continue to optimise the data residency solution, we will introduce additional improvements to enhance the customer experience. Some of these proposed improvements may require additional actions from app developers. Below are a few areas we plan to explore in the coming 12 months:
- Data storage classification for apps: We plan to introduce new parameters within the app descriptor/manifest for app developers to declare if an app stores / does not store in-scope data, and if an app stores in-scope data within the host products supporting data residency (Jira/Confluence). This will provide customer admins with a more complete view on admin.atlassian.com regarding their installed app’s data residency status.
- Email notifications for app migrations: Upon initial release of Connect realm migration service, app migrations status will not be included in the existing email notifications for customers. We plan to add app migration information to email notifications in future updates.
- Tenant pinning API: We are currently looking into feasibility of providing an API for Marketplace Partners to identify the current pinned location of a tenant’s host product.
- App data migration estimations: We plan to introduce a new API for apps to provide a time estimate for an app data migration for a particular tenant/installation. This will aim to align the customer experience between the host product migrations and app migrations, and ensure the customer has the relevant information to plan for any required downtime.
- Realm pinning persistence for app re-installs: We are looking into updating the existing realm pinning flows to enable realm persistence during an app re-install. This will ensure existing app data sets remain in the same realm as the app installation. Further, the pinned realm for an installation can only be changed by the customer using the realm migration service.
- Migration of host product and apps within a single window: Upon initial release of the Connect realm migration service, apps will be requested to migrate after the host product has successfully migrated, in its own migration window. However, we are looking into combining the product and app migrations within a single window to make the migration experience simpler.
- Minimising downtime for customers: Initially, apps will be provided a 24 hour maximum migration window to complete the data migration. In future, we will explore reducing this to minimise the amount of downtime for customers. Further, we will explore feasibility of supporting live migrations of app data to eventually eliminate downtime for customers during product and app migrations.
The scope and timelines for these are still being determined. Once we progress with these items, we will reach out to the developer community to gather feedback and keep you updated on target delivery dates.
Where can I go for updates on this topic?
We’ll discuss Forge data residency in the Forge Quarterly Roadmap Webinars, and data residency more broadly in the Marketplace Quarterly Roadmap Webinars. Further updates will also be provided in the Partner Portal (particularly in our trust hub). We will also let you know about documentation changes in the Marketplace change log.