Can't retrieve space mappings & How to get only migrating space-ids

Hi

I am working on the implementation of the app data export for our server plugin. Therefore do I need all the space keys of the migrated spaces. I thought that I could retrieve those over the space mappings (or space mappings in combination with sql calls). I tried to get the space mappings the same way as in the provided example.
This is my code:

public void onStartAppMigration(String transferId, MigrationDetailsV1 migrationDetailsV1) {
        BoundCloudMigrationGateway gateway = accessor.getCloudMigrationGateway();
        try {
            PaginatedMapping paginatedMapping = gateway.getPaginatedMapping(transferId, "confluence:space", 5);
            while (paginatedMapping.next()) {
                Map<String, String> mappings = paginatedMapping.getMapping();
            }
        } catch (Exception e) {
            log.error("Error while retrieving space mappings", e);
        }

But I get always this Bad Request error message (sometimes with code 400 and sometimes with 403):

[INFO] [talledLocalContainer]  -- stepConfig: 8876bf33-1ffb-430c-a607-cd9f96fddc18 | cloudId: b13f4fae-3dd9-4a2d-8c5f-79f2a3edef02 | planName: t9 | stepType: CONFLUENCE_IMPORT | stepId: c328aab0-cff1-4d54-9364-4e05eac1a85f | planId: 57919baf-f10d-45cb-b4ec-55c6dea20603 | cloudUrl: https://phf.atlassian.net | taskName: Migrate space ds | taskId: 4541e3cc-114d-40a0-a244-d31d62205b8f
[INFO] [talledLocalContainer] com.atlassian.migration.agent.okhttp.HttpServiceException: Bad request. Status code: 400, message: 
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.DefaultHttpServiceErrorHandler.accept(DefaultHttpServiceErrorHandler.java:21)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.DefaultHttpServiceErrorHandler.accept(DefaultHttpServiceErrorHandler.java:12)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.ServiceErrorCodeHandler.accept(ServiceErrorCodeHandler.java:36)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.ServiceErrorCodeHandler.accept(ServiceErrorCodeHandler.java:13)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.HttpService.callImpl(HttpService.java:122)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.HttpService.call(HttpService.java:97)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.okhttp.HttpService.callJson(HttpService.java:54)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.DefaultAppMigrationServiceClient.getMigrationMappingByPage(DefaultAppMigrationServiceClient.java:129)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.DefaultPaginatedMapping.next(DefaultPaginatedMapping.java:37)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.tracker.BoundPaginatedMapping.next(BoundPaginatedMapping.java:17)
[INFO] [talledLocalContainer]   at eps.focuspro.migration.cloudMigration.EdmComponentImpl.onStartAppMigration(EdmComponentImpl.java:50)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.tracker.BoundListenerImplV1.onStartAppMigration(BoundListenerImplV1.java:22)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.AbstractCloudMigrationRegistrar.lambda$startMigration$4(AbstractCloudMigrationRegistrar.java:107)
[INFO] [talledLocalContainer]   at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
[INFO] [talledLocalContainer]   at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1556)
[INFO] [talledLocalContainer]   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
[INFO] [talledLocalContainer]   at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
[INFO] [talledLocalContainer]   at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
[INFO] [talledLocalContainer]   at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
[INFO] [talledLocalContainer]   at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
[INFO] [talledLocalContainer]   at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
[INFO] [talledLocalContainer]   at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
[INFO] [talledLocalContainer]   at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
[INFO] [talledLocalContainer]   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
[INFO] [talledLocalContainer]   at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
[INFO] [talledLocalContainer]   at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.AbstractCloudMigrationRegistrar.startMigration(AbstractCloudMigrationRegistrar.java:90)
[INFO] [talledLocalContainer]   at com.atlassian.migration.app.DefaultRegistrar.startMigration(DefaultRegistrar.java:35)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.impl.AppMigrationDevelopmentService.lambda$rerunAppMigrationForPlan$0(AppMigrationDevelopmentService.java:43)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.lambda$on$0(PluginTransactionTemplate.java:10)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.jpa.impl.DefaultPluginTransactionTemplate.on(DefaultPluginTransactionTemplate.java:35)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.on(PluginTransactionTemplate.java:9)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.read(PluginTransactionTemplate.java:20)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.impl.AppMigrationDevelopmentService.rerunAppMigrationForPlan(AppMigrationDevelopmentService.java:36)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.ProgressTracker.lambda$succeedPlan$7(ProgressTracker.java:215)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.lambda$on$0(PluginTransactionTemplate.java:10)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.jpa.impl.DefaultPluginTransactionTemplate.on(DefaultPluginTransactionTemplate.java:35)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.on(PluginTransactionTemplate.java:9)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.write(PluginTransactionTemplate.java:28)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.ProgressTracker.succeedPlan(ProgressTracker.java:206)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.planning.StepCoordinator.completePlan(StepCoordinator.java:181)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.planning.StepCoordinator.lambda$null$1(StepCoordinator.java:168)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.lambda$on$0(PluginTransactionTemplate.java:10)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.jpa.impl.DefaultPluginTransactionTemplate.on(DefaultPluginTransactionTemplate.java:40)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.on(PluginTransactionTemplate.java:9)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.store.tx.PluginTransactionTemplate.write(PluginTransactionTemplate.java:28)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.planning.StepCoordinator.lambda$completeStep$2(StepCoordinator.java:122)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.util.logging.LoggingContext.executeWithContext(LoggingContext.java:147)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.logging.LoggingContextBuilder.execute(LoggingContextBuilder.java:121)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.planning.StepCoordinator.completeStep(StepCoordinator.java:121)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.StepSchedulerService.lambda$null$3(StepSchedulerService.java:201)
[INFO] [talledLocalContainer]   at java.util.Optional.ifPresent(Optional.java:159)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.StepSchedulerService.lambda$runStep$4(StepSchedulerService.java:199)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.util.logging.LoggingContext.executeWithContext(LoggingContext.java:147)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.logging.LoggingContextBuilder.execute(LoggingContextBuilder.java:121)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.StepSchedulerService.runStep(StepSchedulerService.java:197)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.StepSchedulerService.runScheduledStep(StepSchedulerService.java:193)
[INFO] [talledLocalContainer]   at com.atlassian.migration.agent.service.stepexecutor.space.SpaceImportExecutor.runJob(SpaceImportExecutor.java:144)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.doRunJob(JobRunnerWrapper.java:117)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.lambda$runJob$0(JobRunnerWrapper.java:87)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:84)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:68)
[INFO] [talledLocalContainer]   at com.atlassian.confluence.impl.schedule.caesium.JobRunnerWrapper.runJob(JobRunnerWrapper.java:87)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeLocalJob(CaesiumSchedulerService.java:402)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:380)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
[INFO] [talledLocalContainer]   at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
[INFO] [talledLocalContainer]   at java.lang.Thread.run(Thread.java:748)

Does someone know what the problem could be?
(The line ‘EdmComponentImpl.java:50’ is the ‘while (paginatedMapping.next()) {’)

Kind Regards
Philipp

Hi @PhilippFrank,

Sorry to see you’re getting this result. I’ve had a look at the logs and I can see for the HTTP 400 error you’re getting, somehow our code is calling this URL (I’ve removed the transferId)

/api/migration/mapping/<transferId>/page?pageSize=-1&namespace=confluence%3Aspace

I don’t quite know how the pageSize of -1 got in there so I need to do some testing to confirm what’s going on and get the team to look into it.

For the HTTP 403, that’s because you didn’t have the MIGRATION_TRACING_PRODUCT scope listed.

For now, I can see the HTTP 400 happened twice for your cloud site in the last 3 days. Are you doing your migration through the UI or are you using the REST endpoint to restart the just the app migration? Can you reproduce this?

Regards,
James.

Hi @jrichards

Thank you for your fast response. The pageSize=-1 is my fault, don’t worry (played with the parameter). I added the MIGRATION_TRACING_PRODUCT scope and can get the mappings now. Thanks for the help.
I see that when I retrieve the confluence:space mappings, I get all spaces ever migrated to the cloud instance. Is there a way how I can only get the spaces migrated in the running migration plan?

Best Regards
Philiipp

Hi @PhilippFrank,

Thanks for the update. We were wondering how the -1 got in there. At least it shows our endpoint validation and logging is working.

The reason you get all Spaces is because customers on large instances may spend a few weeks doing the migration and so we provide a list of all the spaces from previous migrations. Then app migration at the end will have a historical view of all the migrations and can process all the required data.

Is there a specific reason you want just the most recent spaces?

Regards,
James.

Hi @jrichards

When a customer makes a partial migration and only migrates one or two spaces, then we shouldn’t update our space settings in cloud for all migrated but only for the most recent ones.
My colleague pointed out that there also are PaginatedContainers which are used as we expect (only show spaces that are migrated with the current plan).
=> therefore:

  • want spaces only in the current migration plan → use PaginatedContainers
  • want all spaces that were migrated → use PaginatedMappings

Thanks for your help. Now all works as should :grinning:

Kind Regards
Philipp

1 Like