Incremental data retrieval?

Cloud Migrations
1

Hi. Sorry to be witless, but would you please elaborate on this statement from the Getting started page?

“Recommendation : Where possible, design your data retrieval process to be incremental, and consider merging data. Jira/Confluence admin users are likely to run multiple migrations that target the same cloud site.”

I’m not clear on what increments the statement refers to.

I think the ‘merging data’ might refer to somehow incorporating server data changes into data on the cloud side, but I’m not certain. In any case, what kind of merge is anticipated there?

Thanks for your patience.

1 Like
2

Hi @david.pinn,

As I don’t know your app, so I can only give some generic ideas about transfer of data. But generally we think that when send send data to your cloud app, keep a record in server of the data sent (like a log or similar with a datestamp) and then on subsequent migrations you can send new data, or changed data since the last migration.

When it comes to merging data, for example, if you have a configuration item, for example a Yes/No flag and you send the previous value to cloud and record that it has been sent. If that value changes you need to record the change and then determine what your app will do. It can either

  1. Update the value in cloud ignoring the cloud value
  2. Warn the user that the value has changed and is different in cloud
  3. Not update the value

It really depends on what the data is and can it be safely overwritten.

Hope this helps.

Regards,
James.

3

Thanks James. That was helpful.

Are you aware of people building user interfaces on the Cloud side to manage the merging of server data, or to manage migrations in general?

4

Hi @david.pinn,

I’m not really aware of apps that manage it. If anyone else has some suggestions here, that would be great.

Regards,
James.

5

Just so you know, the mention of ‘merging’ rings a tiny little alarm bell in the back of my brain. Merging is most commonly associated with synchronization, and synchronization is one scary can of worms to open. This is still a migration tool we’re building, right?

6

Hi @david.pinn,
The migration assistants allow our customers to do the following:

  • Migrate an entire server instance to the cloud
  • Merge multiple server instances into a single cloud site
  • Migrate some projects of their server instance now (for one department), and some others (for another department) later.

For that reason, we enable our customers to run “project by project” or “space by space” migrations instead of a full site import.
Hence, app data migration is expected to cover these use cases, but we don’t expect your app to sync server changes of already migrated projects. In other words, we want to allow incremental migration, not a full synchronization between server and cloud.

Regards,
David Almeida.

7

Hi @DavidAlmeida ,

That the clarification I was after. Thanks very much.

David