Search Content by CQL REST API - Cursor Expiry

Confluence Development Confluence Cloud
,
1

In the following api,
https://developer.atlassian.com/cloud/confluence/rest/api-group-content/#api-api-
content-search-get
_links.next has cursor. When does this cursor expire?
For ex - https://idc-oob.atlassian.net/wiki/rest/api/content/search?next=true&cursor=_sa_WzE2MTc0Mjk3MTAwMDAsIlx0MjU4ODY3MjEgJDxNNFVvQTJqZ1ksTVZfcHFTWScgY3AiXQ%3D%3D&expand=body.view,restrictions.read.restrictions.user,restrictions.read.restrictions.group,space,history,ancestors,history.lastUpdated,history.contributors.publishers.users,children.attachment&limit=15&start=15&cql=space+!=+IDC+order+%20by+lastModified+asc&status=current

2

@ShwetaGulati the cursor returned by this API doesn’t expire.

3

Thanks @JatinChopra ,
So how is the result order guaranteed . If I have order by clause in CQL.
Lets say I order by lastModified. I have fetched some 10 resultsets. It so happens that I fetch 11th resultset after 2 days and in that time , some of the pages fetched were modified again . What will be the behaviour now?
Will it also consider the new modifications?
Or not?

4

Great question!
The cursor returned by Search Platform is a stateless cursor that doesn’t expire. This comes at the cost of predictable/consistent pagination, as records get added/moved/deleted over time. As such, to improve consistency of pagination/user experience, we encode the content ids of previous page for deduplication.

Does this answer your question?

5

So, what I understand is records won’t be duplicated in two subsequent pages.
And do you encode content ids in cursor?

6

@JatinChopra ,

Is there any loss of records ?
Also the cursor doesn’t seem to have entry of all the previous items returned till then
where do you store the previous item IDs ?

7

Hello @ShwetaGulati, I’m an engineer in the Search Platform, and would love to assist with your query.

We do encode the content ids in the cursor to improve the pagination consistency. The cursor format is opaque to consumers, it is highly compressed binary blob of a few details along with the entity ids for this purpose.

As such there isn’t a loss of previous item as long as the consumer calls the api with cursor, however as you can imagine, the stateless cursor is a live cursor, so lets say another user in the meantime removes/changes a record from the content store, the previous page of results will reflect the latest state of the store (which in contrast is different from stateful cursors, which take a snapshot of records, thus resource intensive, and hence expire).

Feel free to reach out if you have further questions, have a good day.

2 Likes
8

Hi @raz
Thanks for this.
Help me understand this.
What you really mean is if I go to previous page through _links.prev , I will get pages with same content ids. Its just that there updated state would be there (in case updates have happened) .

  1. So, is it reasonable to assume there will be no duplicates across all pages?
  2. What I also understand is the time you create cursor you are encoding entity ids in the cursor with their order and hence the order is becoming predictable. Confirm if it is right?
  3. If lastModified > <some_time> order by lastModified , if a change is made in page already retrieved . that content id will not appear again in the resultset. Is this a fair assumption? Basically content ids will not move here and there.