RFCs are a way for Atlassian to share what we’re working on with our valued developer community.
It’s a document for building shared understanding of a topic. It expresses a technical solution, but can also communicate how it should be built or even document standards. The most important aspect of an RFC is that a written specification facilitates feedback and drives consensus. It is not a tool for approving or committing to ideas , but more so a collaborative practice to shape an idea and to find serious flaws early.
Please respect our community guidelines: keep it welcoming and safe by commenting on the idea not the people (especially the author); keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!
Summary of Project:
To help partners more easily meet customer data residency needs, we are enabling data residency support for apps that choose to exclusively use Forge Hosted Storage (i.e. Storage API). This will make supporting data residency much easier for partners with apps that exclusively use Forge Hosted Storage for in-scope data, as these apps will automatically support data residency.
Data residency for Forge Hosted Storage will be available for all the same locations as Jira and Confluence data residency at launch, and Forge will support new locations on the same timeline as other Atlassian products.
- Publish : 10 October 2023
- Discuss : 27 October 2023
- Resolve :
10 November 202317 November 2023
Problem
Data residency gives organisations more control over their data by helping to ensure any private data is stored within the location they trust. Many organisations care about data residency and see it as a requirement to do business with a SaaS vendor. Data residency can also make it easier for companies to meet regulatory obligations.
If customers work in a regulated industry like finance, government, or healthcare, data residency may be a necessity for operating in a cloud environment. In fact, according to our customer surveys, 80% of Atlassian enterprise cloud customers say that apps supporting data residency is very or extremely important to them.
That said, independently supporting multiple locations takes extra work and can come at an additional cost to Marketplace Partners. Such investments must be weighed against other improvements to your app, which often leads partners and developers to put off adoption of this important feature.
Proposed Solution
To help partners more easily meet customer data residency needs, we are enabling data residency support for apps that choose to exclusively use Forge Hosted Storage. These apps will be able support data residency for in-scope data with less separate investment (compared to supporting data residency for external databases).
Forge will take care of the hosting, pinning, and migration of hosted data between supported locations, so partners can focus on building a high quality app for their customers.
The proposed solution is designed to work harmoniously with product data residency and consequently is based on similar concepts and terminology. (Refer to Understand data residency for more information about product data residency).
How will it work?
Upon release, data residency will be available as an inherent feature of Forge Hosted Storage. With this change:
- Any data that is stored in Forge Hosted Storage will be automatically migrated to the same location as the host product (starting at the time of Beta release).
- All new installations of an app using Forge Hosted Storage will have data stored (for relevant tenant) in the same location as the parent product.
- When a customer requests the parent product to be moved to a new location, the relevant Forge Hosted Storage partition will move along with the host product data.
- Forge apps that exclusively store in-scope End User Data within Forge Hosted Storage will automatically be shown under “pinned” in the customer data residency UI (meaning they have pinned stored data to the same location as the parent product).
- If the parent product is not pinned, Forge apps will remain in the Global location along with the parent product data.
Forge Hosted Storage
-
Pinning location: App partition (for relevant tenant) will be pinned/migrated to same location as the host product
-
Who determines the location: Customer admin determine the pinning location of the host product (and hence the location of the relevant Forge app partition). If the parent product is not pinned, Forge apps will remain in the Global location along with the parent product data.
Forge Hosted Compute
-
Pinning location: Forge Hosted Compute will be provisioned and operational in all Atlassian supported locations to ensure acceptable performance (when Forge Hosted Storage is multi-location enabled). As with Jira and Confluence data residency, data-in-transit will not be pinned to any particular location.
-
Who determines the location: Atlassian will determine the location of the hosted compute to ensure acceptable level of performance.
Note: Like Jira or Confluence data residency, only at-rest data will be pinned. Data-in-transit (up to 30 days), app logs, cached content and user account information will not be pinned. If the data residency scope changes for the parent product data residency, we will evaluate similar changes for Forge.
Determining the app pinning status
An app’s pinning status is a status field presented to customer admins in the admin.atlassian.com data residency UI, in order for admins to determine which apps are ‘pinned’ to their requested location.
A Forge app’s pinning status (for the relevant tenant) criteria will be determined based on the app’s manifest declarations:
- App uses Forge Hosted Storage: “Storage:app” scope is defined in the app manifest
- In-scope EUD (End User Data) is stored exclusively in Forge Hosted Storage: In-scope EUD is NOT egressed to remote storage (Note: Partners will need to define and document what is ‘in-scope’ for their app’s data residency policy)
New parameters will be available within the Forge manifest to allow developers to declare additional information that will determine the app’s pinning status:
operation
ofremotes
(e.g. storage, compute or other)- If the
remotes
is used forstorage
, if it storesin-scope EUD
(as defined in the app’s data residency policy) or not
Here are some examples of the manifest declarations that determine the app’s pinning status:
Example 1: App is shown as ‘Pinned’
Permissions:
Scopes:
- "Storage:app"
- Forge Hosted Storage scope is used
- No
remotes
is defined
Example 2: App is shown as ‘Pinned’
Permissions:
Scopes:
- "Storage:app"
remotes:
- key: jsonplaceholder
baseUrl: "https://jsonplaceholder.typicode.com"
operations:
- storage
storage:
in-scope EUD: false
- Forge Hosted Storage scope is used
- Defined
remotes
is used forstorage
, however, not for storage ofin-scope EUD
Example 3: App is shown as ‘Pinned’
Permissions:
Scopes:
- "Storage:app"
remotes:
- key: jsonplaceholder
baseUrl: "https://jsonplaceholder.typicode.com"
operations:
- compute
- Forge Hosted Storage scope is used
- Defined
remotes
is used forcompute
Note: in-scope EUD
is only required to be defined if operation
= storage
Example 4: App is shown as ‘Not Eligible’
Permissions:
Scopes:
- "Storage:app"
remotes:
- key: jsonplaceholder
baseUrl: "https://jsonplaceholder.typicode.com"
operations:
- storage
storage:
in-scope EUD: true
- Forge Hosted Storage scope is used
- Defined
remotes
is used forstorage
, AND storesin-scope EUD
(i.e. in-scope EUD is not exclusively stored in Forge Hosted Storage)
Example 5: App is shown as ‘Not Eligible’
Permissions:
Scopes:
- "Storage:app"
remotes:
- key: jsonplaceholder
baseUrl: "https://jsonplaceholder.typicode.com"
- Forge Hosted Storage scope is used
remotes
is defined, however itsoperation
is not defined (therefore, it cannot be determined if in-scope EUD is exclusively stored within Forge Hosted Storage)
Things to consider:
- For the manifest changes listed above, what constitutes a major or minor version bump for an app depends on if the scope of
remotes
is increasing or decreasing. If it’s increasing the scope of what is being done with theoperation
, such as adding a newoperation
or changingin-scope EUD
totrue
, this would constitute a major version bump. If it’s decreasing the scope of what it’s used for, such as reducing the number ofoperation
, adding a classification to a remote (where previously had no classifications) or changingin-scope EUD
tofalse
, that would be a minor version bump. - Defining the
operation
for remotes is not mandatory for the initial release. However, if your app hasremotes
defined, without anoperation
andin-scope EUD
(when applicable) defined, the app will not be shown to customers as ‘pinned’ - If you define the
operation
as storage, it is mandatory to declare if it does/doesn’t storein-scope EUD
. Validation checks for app deployment will be added. - If a new app version includes
remotes
that is now storingin-scope EUD
(as part of the change), the pinning status of the app may change
Changes for external permissions
As part of external permissions, you can declare URLs that your app should have access to as part of the external.fetch
section of the manifest. As part of these calls, an app can potentially be egressing data for storage. In order to identify these scenarios, we are adding support to pass a key to a remote, as well as the existing system today. The changes would look like:
From:
permissions:
external:
fetch:
backend:
- '*.example-dev.com'
To:
permissions:
external:
fetch:
backend:
- remote-backend
remotes:
key: remote-backend
baseUrl: '*.example-dev.com'
operations:
- other
Note: In future, we will explore extending data residency capabilities to Forge remote storage. Depending on the exploration outcomes, the status criteria may be updated to accommodate for apps storing in-scope EUD
in remote storage services with data residency capabilities.
FAQ
How will multi-location hosted compute be enabled?
Multi-location hosted compute will be automatically enabled for all Forge apps by the time Forge Hosted Storage data residency is rolled out to production. This will ensure there is acceptable level of performance when Forge Hosted Storage is multi-location enabled.
However, in order for it to take effect, a redeployment of your app will be required. This will be a minor version update. We will make an announcement and give you advanced notice before you need to take this action.
Additionally, as Atlassian continues to release support for new locations, subsequent redeployment of your app will be required for these new locations to take effect. If you have not redeployed your app, the app and its users may experience performance issues due to cross-location latency.
NOTE: Automated redeployment is something we are still exploring, and will not be available in the initial releases. We are assessing the level of improvement this will achieve for developer experience and determining its priority.
What will the customer experience look like?
The customer experience within AdminHub for data residency of cloud apps will continue to be a unified experience between Forge and Connect apps. The biggest change that will result from this project will be that for Forge apps using hosted storage, the relevant app partition will be migrated together with the host product, within a single migration window determined by the customer admin. Admins will be able to assess which apps will be moved together with the product, as well as see an updated pinning status of the app post-migration.
What will partners need to do?
Partners will be responsible for defining, documenting, and communicating what data is in-scope for data residency for each of their Forge apps. In-scope data will include any data stored in Forge Hosted Storage.
Partners with apps that use Forge Hosted Storage will be encouraged to take a few steps near the Beta release, including:
- Define and document for customers what is “in-scope” for data residency (which should include all data stored in Forge Hosted Storage).
- Update the manifest, in order to accurately show the pinning status to customer (i.e. declare that the app does not egress in-scope End User Data for storage purposes). Any remotes defined should have the operation and in-scope EUD (when applicable) details declared. A redeployment of the app will required.
- Update the Privacy & Security tab for any app that exclusively uses Forge storage to show customers that the app supports data residency in the same locations as Atlassian for any in-scope data.
What should partners do if they don’t want to support a location?
If your app uses Forge Hosted Storage but you don’t wish to support one of the Atlassian supported locations for your app data residency, you can stipulate this in your terms with customers.
While there is no technical limitation to a customer moving their data to an Atlassian supported location, you can include stipulations about locations where customers can store data in your end user terms or any other customer contracts at your own discretion. That said, it’s important to note that Atlassian is not a party to your contracts with customers and cannot enforce the terms by prohibiting the customer from storing data in certain locations.
Asks
While we would appreciate any reactions you have to this RFC (even if it’s simply giving it a supportive “Agree, no serious flaws”), we’re especially interested in learning more about:
- What do you think about installed apps (that meet pinning criteria) being automatically shown as pinned to customers? Are there any use-cases for an app (that meets pinning criteria) to be not shown as ‘pinned’?
- What level of visibility do you anticipate needing around migration requests and hosting location of hosted storage/compute, and why?
- What do you think about having to provide additional declarations for an app’s remotes, in order to accurately determine the app’s pinning status?
- What are your thoughts on if we make operation declarations mandatory for any definedremotes in the future (i.e. app deployments will be blocked if operation is not defined)? What are the ways this may impact you that we may not have anticipated?
- What do you think about having to redeploy your app every time Atlassian supports new locations? Are there other ways this may impact that we haven’t anticipated?
- Considering the app pinning status may change depending on the manifest declaration updates for a new app version, how do you currently communicate any such changes to your customer admins that could affect their compliance requirements? Are there other ways this may impact that we haven’t anticipated?
We are currently targeting Q1 CY24 for release of Forge Hosted Storage data residency Preview (Customer Beta). You can track this project in Atlassian public cloud roadmap, Forge public roadmap and FRGE feedback board.
Thanks for your feedback!