Deprecation of `start` parameter in REST API

Hey,

The start parameter of some content search APIs is going to be deprecated very soon and I am still a bit confused of how we should address some common use cases. The start parameter will be replaced by a cursor and it is recommended to use the next and previous links returned by the endpoint to scroll through the whole result. However, I do not see how that is feasible in some use cases.

Let’s say I have a list of 1000 results and I want to display them in 20 pages, 50 results on each page. A user wants to see page 10 directly. How do I get the results 451 to 500? Will there be a way to skip result 1 to 450? Or do I have to actually fetch results 1 to 450 and discard them just to get the cursor to be at result 451?

I already asked this questions some weeks ago in the according thread in #Confluence:confluence-cloud-changelog, along with questions of other vendors, but got no response from Atlassian so far. I think this is a common use case when paginating results and we definitely have it. As the deprecation deadline is approaching, can anyone help me on that? Maybe someone from Atlassian? Or anyone else who is facing this issue?

Cheers,
Peter

2 Likes

I think that will be the only way unless Atlassian updates the API again to support your use case. Of course loading all search results aggressively is not a nice thing to do as it puts more stress on Atlassian’s servers and will also be a worse experience for the user. But if this is a core feature that your users rely on then I think you’ll have no other choice unfortunately. :confused:

Cheers,
Sven

1 Like

I am still interested in hearing Atlassian address these use cases as well, but it’s been all crickets since those questions were asked.

2 Likes

Hey Sven,

Thanks for your impression on that. I don’t think that our users rely heavily on this kind of pagination and if there is no viable solution we will change how we present the results to the user anyway. I just thought that this must be a common use case and the only solution I could think of is rather a workaround. Maybe it is not so common, but still there were also some questions from other vendors regarding this deprecation which have not been addressed. So I thought I’d raise this here again, because as far as I experienced here Atlassian is responsive to some extent.

Cheers,
Peter

2 Likes

Thanks for raising this @pete,

My understanding is that the change to cursor based iteration through search results is due to a change to a graph storage mechanism where it isn’t feasible to index into the middle of search results.

Picking up on your note about being not “feasible in some cases”, how often do your users skip directly into a specific page rather than iterating through them? Do you have any analytics on this? Can you change the design to accommodate the new API such as an infinite scroll based UI and maybe suggesting the user refine their search if it looks like there are a huge number of results?

Regards,
Dugald

Thanks for coming back to me @dmorrow,

I appreciate your suggestions and we are preparing to adapt our design to incorporate this change. We do not have analytics on that, but as you suggested it is probably not heavily used and reasonable to assume a refined result set.

However, could you maybe also address the other questions in the according #Confluence:confluence-cloud-changelog thread? What about other endpoints with result sets, will the start parameter still be supported there? Is there a way to query pages in parallel?

Cheers,
Peter

2 Likes

Hi @pete,

I’ll see if I can draw attention to those questions from the relevant Confluence experts.

Regards,
Dugald

2 Likes

A few developers have raised concerns about the change in how the search REST API pagination works. We’re preparing examples that show how to use the new cursor-based method and we’ll post by tomorrow at the latest. In the meantime, to give you more time to update your code, we are extending the deprecation date to July 24, 2020 .

To make it easier to test your code against the API in its final form, we will roll out the change to the Confluence Cloud: Developer-First Rollouts for Ecosystem App Developers (formerly known as Ecosystem Beta Group) on July 20, 2020.

2 Likes

Hi all,

Please see this thread for the latest updates on this topic: Upcoming changes to modernize search REST APIs