REST API "/pagehierarchy/copy" causing multiple copies to be created

In a connect app, I am copying the pages of one space (a template) to another (user created) space upon user request and I am getting multiple copies of 1 particular page tree in the hierarchy. I’ve also noticed that if I don’t cap the wait time for the process to complete at 30-60 seconds, I can be waiting for a very long time, even though I clearly see pages have been copied very quickly. When I check all of the long tasks, I notice some have never completed and still look like they are trying to complete, even after hours (maybe longer) of not doing this particular task, and all of them are the copies for this particular page tree to other previously created (and possibly deleted) spaces. Any idea what can be going on here? I only copy each page once in the code below.

The steps I take:

  1. Get the subpages of the template space at the root level via:

/rest/api/space/TEMPLATESPACE/content
with options: ‘depth’:‘root’,“expand”: “children.page”

  1. Iterate over the template subpages and call:

‘/rest/api/content/’ + template_subpages[i].id + ‘/pagehierarchy/copy’,
with options:
{
“copyAttachments”: true,
“copyPermissions”: true,
“copyProperties”: true,
“copyLabels”: true,
“destinationPageId”: project_space_homepage.id,
}

  1. Process the response to get the long task id and wait for it to complete via:

    let copy_done = false;

    while(!copy_done)
    {
    let long_task_status = JSON.parse(data.body);
    var data = await AP.request(
    {
    url: long_task.links.status,
    error: function(error)
    {
    console.log("Error getting project homepage ", error)
    return;
    }
    });

    if(long_task_status.percentageComplete >= 100)
      copy_done = true;
    

    }

Here’s and example of what is created:

Confluence_Error

3 Likes

For anyone else who may run into this issue, this only seems to be a problem for me when ‘/pagehierarchy/copy’ has option “copyProperties”: true. If this is false, the copy happens as intended.

1 Like

Hello @NathanMiller:

We call this the “5 copy, takes a long time” bug. There is, apparently, some bad error handling/retry code in the cph API implementation. These days we most commonly run into it when copying attachments. In those cases when we disable attachment copying we don’t see the problem.

One of the original bugs used to make changes in an effort by Atlassian to solve the “5 copy” problem: https://jira.atlassian.com/browse/CONFCLOUD-67045
That bug found bad error handling in Atlassian. In that bug it was determined that a “partial” failure could cause the copy process to be retried 5 times:
" The root cause in this particular case (from the demo space you provided) is that there is an invalid property set on a particular piece of content.
The symptom is that the copy-page request partially fails, so that data is copied but is retried, and therefore we end up with 5 copies.
There are a variety of improvements for us to implement, but we can also do quick workarounds by directly modifying the database for specific problematic cases."

If you have a reproducer I’m sure that Atlassian would love to have it. We have an open DEVHELP ticket with them that you can reference if you open a ticket: DEVHELP-4098.

If Atlassian is watching, please respond to our ticket… it has been open for a few months with no response. This bug is being reported by multiple customers of ours.

Thanks!

2 Likes

@TureHoefner Thanks for providing detailed repro steps and a data set. I’ve assigned this issue and will have them look into it.

/cc @rwhitbeck

1 Like

Awesome. Thanks @nmansilla!

1 Like