RFCs are a way for Atlassian to share what we’re working on with our valued developer community.
It’s a document for building shared understanding of a topic. It expresses a technical solution, but can also communicate how it should be built or even document standards. The most important aspect of an RFC is that a written specification facilitates feedback and drives consensus. It is not a tool for approving or committing to ideas, but more so a collaborative practice to shape an idea and to find serious flaws early.
Please respect our community guidelines : keep it welcoming and safe by commenting on the idea not the people (especially the author); keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!
Summary
Data Residency for Forge Remote aims to enable Forge remote apps to meet data residency requirements by supporting remotes with region-specific data storage capabilities. This project responds to the increasing need for compliance with regional data residency regulations, as well as customer demands for data storage localisation to improve performance and ensure privacy.
We will address two aspects of Data Residency for Forge Remote, realm pinning and realm migrations.
Publish: Jul 29, 2024
Discuss: Aug 11, 2024
Resolve: Aug 18, 2024
Problem
App partners are looking to adopt Forge Remote; however, many of these apps require storage of user data, making data residency a critical requirement for customers. Customers need data residency for reasons such as complying with local privacy and security laws or reducing latency. Currently, Forge Remote does not support data residency, preventing these apps from transitioning to Forge Remote and meeting customer expectations and regulatory requirements.
Proposed Solution
The solution will allow Forge apps to specify multiple remotes, each with defined region-based URLs for data storage. Only remotes declaring storage operations with EUD will be required to support region-based URLs, providing clear communication to customers about data residency and storage locations.
An example manifest design illustrating the proposed changes is provided below, closely mirroring the Connect method for handling data residency. This design choice facilitates a smooth and straightforward transition from Connect to Forge Remote. In the example manifest, each remote includes a default base URL and region-based URLs supported by the app. A separate migration module is defined, directing migration hooks to a specific remote. This setup allows for flexibility in data management and compliance with data residency requirements.
....
permissions:
external:
fetch:
backend:
- remote: forge-remote1
remotes:
- key: connect-remote
baseUrl:
default: "https://mr-connect.com"
us: "https://us-mr-connect.com"
eu: "https://eu-mr-connect.com"
au: "https://au-mr-connect.com"
operations:
- storage
- fetch
storage:
inScopeEUD: true
- key: forge-remote1
baseUrl:
default: "https://ms-forge1.com"
us: "https://us-ms-forge1.com"
eu: "https://eu-ms-forge1.com"
au: "https://au-ms-forge1.com"
operations:
- storage
- fetch
storage:
inScopeEUD: true
- key: dare-remote
baseUrl:
default: "https://dare-migration.com"
us: "https://us-dare-migration.com"
eu: "https://eu-dare-migration.com"
au: "https://au-dare-migration.com"
operations:
- fetch
modules:
migration:
dataResidency:
remote: dare-remote
migrationPath: "/migration-hook" //This can be optional. Hooks could directly go to the baseUrl
maxMigrationDuration: 3
app:
connect:
key: mr.connect
remote: connect-remote
...
Considerations
- All remotes with the storage operation defined and
inScopeEUD
true
must support the same region-based URLs to be eligible for realm pinning. - All remotes egressing
inScopeEUD
data need to support the same regions to be eligible. - If any remote doesn’t have
storage
andinScopeEUD
but they declare region based urls for compute purposes, they will need to support all the regions that are declared in storage remotes - Apps storing data both on Atlassian (in hosted storage) and externally via a remote will be eligible for realm pinning, provided the remote supports region-based URLs.
- A new option will be available in the Privacy and Security tab to inform customers of this setup.
- Changes to the Privacy and Security tab will ensure customers are informed about what each partner constitutes in-scope and out-of-scope EUD.
- Apps storing EUD on Atlassian with remotes not storing EUD will continue as usual, with hosted EUD pinned to the product’s region.
- Migration hooks will support Forge Invocation Tokens (FIT) and not Connect JWTs authentication.
- Note: Your data residency migration path, does not have to point to a separate remote as shown in the example.
Major Version changes
We have identified a number of actions that would require a major version upgrade for customers. I can empathise that the current major version upgrade experience is a pain point in Forge currently. Atlassian are currently looking into the problem space, however, we will not be addressing that issue with this piece of work.
The actions that will require a major version upgrade are:
- Adding
regionBaseUrls
- Changing existing remote
baseUrl
orregionBaseUrls
- Changing the
operations
of an existing remote and addingstorage
withinScopeEud
true - Removing
regionBaseUrls
Out of scope
We have a number of features we would like to include in this pass of Data residency, however, for this first iteration we have a number of features which we will not be addressing in this immediate piece of work.
- Apps using direct URLs through external → fetch will not be eligible for pinned status due to the support of wildcard URLs
- Multi-environment support in a single manifest.
- Realm Persistence feature. We will be addressing this feature as a fast follow.
Asks
While we appreciate any feedback on this RFC, we are particularly interested in understanding:
- What challenges do you anticipate with the proposed data residency requirements?
- Would this solution meet your requirements in supporting data residency on forge remote?
- Are there aspects of data residency compliance that you feel are not adequately addressed in this proposal?
- Do you have any questions regarding this solution that aren’t addressed here?