Hello @hassandewitt,
Your question is a few months old but if you are still struggling with this or if others found this question after encountering the same issue, the answer is to use the new Confluence v2 API, which is much more performant when paginating over a lot of content.
See Release of v2 Confluence REST API for Pages and Blogposts (Experimental) and Release of new endpoints for Confluence REST API v2