We are using cursor-based confluence search rest API for traversing pages and indexing them. In this approach, we get nextLink in response which is used to make next batch call. During crawling one of batches is failing and crawl is getting terminated. We want to make our system more tolerant and skip these failed batches and continue with crawling.
How can we skip current batch in case if failure when there is no nextLink available?
Any suggestion to overcome this issue is welcome.
We are hitting this issue frequently and need attention asap.