RFC-111 Delta migrations for Apps

General Request for Comments (RFC)
,
1

RFCs are a way for Atlassian to share what we’re working on with our valued developer community.

It’s a document for building shared understanding of a topic. It expresses a technical solution, but can also communicate how it should be built or even document standards. The most important aspect of an RFC is that a written specification facilitates feedback and drives consensus. It is not a tool for approving or committing to ideas, but more so a collaborative practice to shape an idea and to find serious flaws early.

Please respect our community guidelines: keep it welcoming and safe by commenting on the idea not the people (especially the author); keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!

Project Summary

“App Migrations - Delta Migrations for Apps” is an initiative to provide marketplace partners with the ability to support delta migrations for their apps, allowing for smaller migrations with reduced migration downtime. This will be achieved by providing marketplace partners with information regarding the difference between the initial migration and any future deltas and allowing them to decide how to use this information to reconcile the changes.

  • Publish: 26 August 2025

  • Discuss: 16 September 2025

  • Resolve: 30 September 2025

Problem

Atlassian is developing “delta” migrations, which are migrations that run after the initial migration, transferring only the data that has changed since the last migration (the “delta”). These migrations minimise customer downtime and operational pressure by replacing a single large traditional migration (often run on weekends) with multiple delta migrations that offers a wider margin for error and does not block the customer’s server.

However, apps currently lack visibility into data changes between migrations, requiring every app migration to re-migrate all the app’s data during subsequent migrations. Because app migrations must currently be rerun in full, they will not be fully capable of participating in delta migrations.

We have an opportunity to give apps the capacity to fully participate in delta migrations by offering them visibility into data deltas so that they can identify mutations and reconcile the differences with the cloud site. This offers customers less down-time than the traditional migration flow.

Who are we solving for:

For marketplace partners:

  • Allow marketplace partners to choose between a single, large-scale traditional migration, or a set of smaller less-error-prone migrations, based off of deltas from an initial migration. This will:

    • Reduce load on their infrastructure when larger customers migrate

    • Reduce the chances of an error occurring during a migration, thereby reducing the “blast radius” of the error to only the delta migration rather than the whole migration. Meaning that only the delta would need to be rerun rather than the entire migration.

For customers:

  • Offer customers less down-time than the traditional migration flow.

  • Give customers the option for smaller delta migrations with a lower chance of failure

Proposed Solution

Atlassian will provide an updated listener interface for marketplace partners to integrate with, enabling the transmission of a list of modified containers (projects or spaces), and other information since the last delta migration to the cloud site. Marketplace partners can receive these events by utilising the new migrations listeners.

In Your Server App:

Your server app must implement the new listener methods to support delta migrations. Below are the DiscoverableListener interface changes, which would be similar to the changes for the multi-transfer and forge listeners. The new methods are default, so implementation is optional. By default, your app does not support delta migrations.

interface DiscoverableListener {
    // NEW Connect listener methods
    /**
     * Decides whether delta migration is supported for migration 
     * (e.g. for this customer)
     */
    default boolean isDeltaMigrationSupported(DeltaMigrationDetailsV1 migrationDetails) {
      return false;
    }

    /**
     * Perform the delta migration. This functions the same way as 
     * `onStartAppMigration()`, but apps will only migrate the app data that is new
     * or changed.
     */
    default void onStartDeltaMigration(
      DeltaAppCloudForgeMigrationGateway gateway, 
      String transferId,
      DeltaMigrationDetailsV1 migrationDetails
    ) {
    }
  
    // EXISTING Connect listener methods
    void onStartAppMigration(
      AppCloudMigrationGateway gateway, 
      String transferId, 
      MigrationDetailsV1 migrationDetails
    );

    ...
}

// New classes/interfaces used in the above listener definition:
interface DeltaAppCloudMigrationGateway extends AppCloudMigrationGateway {
  /** Get core data containers (ContainerV1) that have changed since last delta migration. */
  PaginatedContainers getPaginatedDeltaContainers(String transferId, ContainerType containerType, int pageSize);
}

class DeltaMigrationDetailsV1 extends MigrationDetailsV1 {
  public Instant getLastDeltaMigrationTimestamp(); 
}

The changes are as follows:

  • boolean isDeltaMigrationSupported(): Atlassian uses this to determine if your app supports delta migrations.

  • void onStartDeltaMigration(): This method is called instead of onStartAppMigration() if your app supports delta migrations. Your app will only migrate new or changed app data. The method passes the following parameters:

Atlassian does not have visibility into each app’s data structures. The intention is that by providing the last delta timestamp and the list of core containers changed since the last migration apps should be able calculate to the delta successfully.

Whether this information is sufficient or not is something we would like your input on.

In your cloud app:

Cloud apps will receive notifications on delta app migration events for each transfers via,

Events for delta migrations will include two additional fields in the event payload:

  • lastDeltaMigrationTimestamp: timestamp of the start of the last delta migration.

  • deltaMigrationTimestamp: timestamp of the start of the current migration as follows.

For Forge apps:

Forge apps can utilise existing Forge app migrations APIs with respective transfer IDs of each transfer.

Sample Forge Event Payload:

{
  "eventType": "avi:ecosystem.migration:uploaded:app_data",
  "transferId": "6c01ac6f-c512-4ef7-8b48-25c1803fe305",
  "lastDeltaMigrationTimestamp": "2025-08-18T15:30:04Z",
  "deltaMigrationTimestamp": "2025-08-21T01:57:32Z",
  "migrationDetails": {
    "migrationId": "403c4f71-a0d1-4a63-97a8-487d18691c46",
    "migrationScopeId": "0ba07dd9-3804-4600-9102-fa6e1efeab08",
    "createdAt": 1723111376499,
    "cloudUrl": "https://your-customer-cloud-site.atlassian.net",
    "name": "Migration Plan Name"
  },
  "key": "e094ca53-3747-4541-b263-0bf7b56a5bca",
  "label": "file-label-you-used",
  "serverAppVersion": "1.0",
  "messageId": "53f88ea7-a2d2-4dd2-9f36-2d8c43401b11"
}

For Connect apps:

Connect apps can utilise existing Connect app migration APIs with respective transfer IDs of each transfer.

Sample Connect Event Payload (for a Jira migration) :

{
    "eventType": "app-data-uploaded",
    "cloudAppKey": "my-cloud-app-key",
    "transferId": "6c01ac6f-c512-4ef7-8b48-25c1803fe305",
    "lastDeltaMigrationTimestamp": "2025-08-18T15:30:04Z",
    "deltaMigrationTimestamp": "2025-08-21T01:57:32Z",
    "migrationDetails": {
        "migrationId": "403c4f71-a0d1-4a63-97a8-487d18691c46",
        "migrationScopeId": "0ba07dd9-3804-4600-9102-fa6e1efeab08",
        "createdAt": 1723111376499,
        "cloudUrl": "https://your-customer-cloud-site.atlassian.net",
        "name": "Migration Plan Name"
        "jiraClientKey": "acb711b8-a878-356e-abbf-1ae1730308a2",
        "confluenceClientKey": "unknown"
    },
    "s3Key": "e094ca53-3747-4541-b263-0bf7b56a5bca",
    "label": "file-label-you-used",
    "serverAppVersion": "1.0",
    "messageId": "53f88ea7-a2d2-4dd2-9f36-2d8c43401b11"
}

Workflow for Delta Migrations:

Delta migrations initially will follow a similar flow to traditional app migrations.

  1. Prior to the listener being triggered, listener.isDeltaMigrationSupported() is evaluated.

    1. If false the existing migration flow is executed with listener.onStartAppMigration().

    2. Otherwise a delta migration is executed with listener.onStartDeltaMigration().

  2. listener.onStartDeltaMigration() can use the aforementioned getLastDeltaMigrationTime() and getPaginatedDeltaContainers() to calculate and migrate their app delta data.

Change Detection

The containers returned by DeltaAppCloudMigrationGateway.getPaginatedDeltaContainers() are only containers that have had core product data changed. This includes core entities such as: issues, pages, comments etc.

We will not be able to track app-specific data changes, including: AO Tables, Files, Custom Fields, Entity Properties etc, we expect our marketplace partners to be responsible for tracking any of those changes by using DeltaMigrationDetailsV1.getLastDeltaMigrationTime().

Asks

We welcome your feedback on this RFC: whether it’s a simple acknowledgment, or more detailed insights. We are particularly interested in your thoughts on the following points:

  • As Marketplace Partners, what information do you need to understand the differences between two migrations for reconciliation? Is providing timestamps of previous delta migrations and projects that contain modified issues enough for you to work out changes in app specific data?

  • As a Marketplace Partner, can you update specific entries in a project’s cloud app specific data without rewriting everything?

  • Beyond the timestamp of the previous delta migration sent to the cloud app during each transfer, what other information would help determine changes between delta migrations for the cloud app?

  • Do you have any extension ideas for the proposed solution? If so, please share your ideas or use cases.

2

Hi Jason,

Thanks for providing the info and also thank you that this has been implemented in Connect and Forge at the same time.

For us, the timestamp is sufficient to determine all changes and we will be able to create a delta easily without rewriting everything.

However, I’d like to highlight that migrations from DC to Cloud are currently not our top priority. I do understand the reason for the change, I do think it’s a superior way to do it and it should be rather straight forward to implement.
But still, given all the other things we have to do with Forge, a working DC to Cloud migration might just be good enough. Longer waiting times for customers are not a deal-breaker.

So while everything seams reasonable we still may not utilize it.

3 Likes
3

Hi Jason.

In our current migration process, we need to edit Confluence pages because macros are not implemented in exactly the same way. To make it easier on ourselves, we currently iterate over pages with a CQL search and mark pages transformed with a custom property (type = page and macro = requirement and isMigrated != true).

My question is this: What happens for custom properties and incremental migrations ? Will the changes to the page in DC create a new page with 0 custom properties ? Will the custom property value be conserved and break our migrations?

4

The hello.atlassian.net link in this RFC is not accessible to non-Atlassians. Can this change detection logic please be published somewhere where app vendors can see it and comment on it?

5

Hi Scott,

Apologies, we’ve fixed up the link, which is a link to the “Change Detection” heading on this page. Thanks!

6

Aside: @JasonWen Is there some kind of Rovo agent that could be run on all of these kinds of links?

This happens time and time again, and seems to be perfect grunt work for AI.

7

Hi @JasonWen,

Thanks for the update. The guidance here is on change detection is admittedly a little bit high level.

  • For example, in Confluence, what is considered to be a “core” container?
  • Are “updated” flags cascaded through any sort of hierarchy?
  • For example, suppose that a user adds an attachment[1] on a comment[2] to a page[3] in a space[4]. Which of these four objects are marked as modified?
  • At the page hierarchy level, is it correct to assume that when a page is modified, the “modified” taint does not cascade up to parent pages or down to child pages?
  • Is a permission update to a page considered to be a change? If so, and if that permission change is of the type that is inherited by its child pages, are those child pages marked as modified?

Thanks!

2 Likes
8

Hello @JasonWen! My first thought on this is that even without delta migrations, users can currently choose to migrate incrementally, e.g. first a project, then another, and so on. As far as I know apps don’t have any way of knowing this migration “scope” at migration time, so we are forced to migrate all data, even if much or most of it will not be used (e.g. because the project or dashboard that it’s related to was not migrated by Jira).

Before adding incremental migrations, maybe it would be better to provide apps with this kind of information?

9

Hi Scott,

Thank you for your input on the Confluence migration path. As we are in the early stages of designing this process, please note that implementation details may evolve. We value your insights and will ensure to keep you informed throughout the development.

The primary goal of the current RFC is to gather feedback on the type of information you would require to identify changes between two incremental migrations. At this stage, we do not plan to include an “updated” flag. Instead, we propose providing a list of modified projects/spaces (containers) along with a timestamp indicating the last successful incremental migration.

Please feel free to share any further thoughts or questions you may have on this approach.

10

Hi @JasonWen

I did not intend to imply that an actual “updated” flag would be made visible to clients of this API. The goal was to clarify exactly which types of containers would be returned by the API. Specifically, this question:

  • In Confluence, what is considered to be a “core” container?

If I read between the lines in your response and I assume that only spaces are considered “core containers” in Confluence, then the answer of what objects will be marked as updated is self-evident.

If, however, pages and other content objects are considered core containers, then there is a question of hierarchy. The “Change Detection” section implies that pages and comments are considered “core entities”, but is that the same thing as a “core container”? (Pages and comments are containers that can have child content too.) Regardless, more clarity would be helpful here.

An answer to this question would help make this really clear:

  • For example, suppose that a user adds an attachment[1] on a comment[2] to a page[3] in a space[4]. Which of these four objects are marked as modified?
11

Hi Coretin,

Currently the incremental migration path for Confluence is still being worked on and no concrete plan has been made yet. However, the incremental flow is expected to remigrate pages so if a page on the server/data centre does not have these custom properties set, it will most likely not be conserved. We will continue to keep you updated on the status of the Confluence incremental migration state and let you know if there are any major changes.

12

Hey @BogdanButnaru,

As a part of AppCloudMigrationGateway in the discoverablelistener interface, we provide market place partners with a class, AppCloudMigrationGateway which gives access to a method called getPaginatedContainers which should return a list of projects, spaces and sites. Hope this helps! Thanks!

1 Like
13

Hi @scott.dudley ,

Yes, you got a great point. Unfortunately there’s no definitive answer I could provide at the moment as the process is still being fleshed out for incremental migrations. However it is expected that changes would be tagged at a page level for now (all changes to child contents will get propagated up).

14

Hey Jason, thanks, I forgot about containers. That could help some.

Is there any way of finding out what other migrations did? For example, if the app has some object that relies on projects A and B (at the same time), and the current migration is only for project B, the app might want to do something different depending on if project A had already been migrated (in an earlier migration) or not.

Also, it would be nice to have information about flags like “migrate all dashboards & filters“ vs. “migrate only those in affected projects”. We can’t tell that only based on the list of projects.

15

Hi @JasonWen.

The idea of a delta migration looks promising, and providing the timestamp appears to be the right approach. From the Zephyr perspective, we have many entities that are not related to any Jira entity, except the project.
From our understanding, we will need to compare the delta migration timestamp with a kind of audit column for each entity (ex, created_at and updated_at). Additionally, it will be necessary to add some kind of record for deleted entities.
During the export phase, it would be necessary to fetch all the data created or updated after the deltaMigrationTimestamp.
Later, during the import phase, there will be three different scenarios to deal with the data.

  1. Import new data. This will be easy, and we can reuse all we have today.
  2. Importing the updated data. This step will be complex. For each updated entity, it would be necessary to fetch the data in our cloud application, using the datacenter IDs. Possibly, some extra work will be required to track back all the entities to store the original IDs from DC on the cloud.
  3. To delete data, a mechanism will be required to track back the entities with the DC IDs.

Long story short, for implementing a delta migration, just a tiny fraction of the current JCMA implementation will be reused. Probably, we will need to rewrite almost everything in the export and import phase. On top of that, additional effort will be required to create a mechanism on DC to track all the deleted entities and introduce additional columns to store the timestamp of each entity when it was inserted or updated.
Of course, we can think about doing that in some incremental fashion, but looking into the big picture, it’s a massive piece of work.

2 Likes
16

Hi @BogdanButnaru,

Great question, Marketplace apps must ensure their data migration logic accounts for whether related projects or objects have already been migrated. This can be done in various ways such as querying the cloud site to check if a specific project already exists in the cloudsite. Is there any other information that you believe would be valuable to retrieve from past migrations?

Regarding granularity of entities that can be migrated, in the context of delta migrations, unfortunately, an exact answer cannot be provided at this time but I’ll keep you posted when there are further developments.

17

Hi @JasonWen
Our app, compatible with Server to Cloud migration, is Projectrak.
During the migration process, we currently handle:

  • Instance-level: custom fields and layouts.

  • Project-level: values and permissions.

On the Server side, our app includes a history mechanism, but we lack a simple way to identify when an entity was created, updated, or deleted.
On the Cloud side, our current logic updates existing data and creates new entities that are found in the Server but missing in Cloud.

We believe that delta migrations would provide real value for users. However, due to the wide range of Projectrak versions we support (to be compatible with different Jira Server/DC environments), implementing this feature would require a significant investment of effort.

It is useful to have a mechanism where the user can explicitly select specific entities to be included in a delta migration. The simplest and most effective approach would be to leverage a lastMigrationDate to determine which entities have changed and need to be included in the delta.