startAt not actually starting at

I’m trying to grab all issues in a project that match my criteria. Up until now, it’s been fine, because we’ve had fewer results than the default maxResults.

Using PHP/cURL, this worked correctly for returning fewer than 50 results:

https://mysite.atlassian.net/rest/api/3/search?jql=issuetype%3DVulnerability%26resolution%3DUnresolved

We just added a number of new issues, and now I’m only getting back the first 50 of 53; here’s the json decoded result (not including the actual issues):

"expand":"schema,names","startAt":0,"maxResults":50,"total":53

I found the documentation indicating that I’d need to paginate my results, and it seems like I should be able to add the startAt param to my querystring.

So I run an initial call to determine how many possible results there are, then determine how many times I’ll need to run the call (total / maxResults). I then do a for loop based on the number of calls, and spit the resulting issues into a separate array for each pass. I’d expect that the first pass would have 50, and the second pass would have 3. But that’s not what I’m seeing. Instead, both passes are returning the first 50 results.

(The code and logic is really messy; bear with me, it’s a first pass.)

// removed authentication headers for the community post; they do exist in my code

// run initial call to get total and figure out number of calls required
$ch = curl_init();
$url = "https://mysite.atlassian.net/rest/api/3/search?jql=issuetype%3DVulnerability%26resolution%3DUnresolved";

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch,CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);
$array = json_decode($result,TRUE);
$searchTotal = $array['total'];
$maxResults = $array['maxResults'];
$numCalls = ceil($searchTotal/$maxResults);
$newStartWith = 0;

error_log("total: $searchTotal"); // this is 53, as expected

for($x = 1; $x <= $numCalls; $x++) {
    error_log("newStartsWith on pass $x: $newStartWith"); 
	// on pass 1, $newStartsWith = 0; on pass 2, $newStartsWith = 50 - this is the expected result
	
    $url = "https://mysite.atlassian.net/rest/api/3/search?startAt%3D{$newStartWith}%26jql%3Dissuetype%3DVulnerability%26resolution%3DUnresolved%20order%20by%20issuekey";
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch,CURLOPT_HTTPHEADER, $headers);

    $result = curl_exec($ch);
    $array = json_decode($result,TRUE);

    $newStartWith += count($array['issues']);
    $tmpVar = $array['issues'];
    foreach($tmpVar as $issue) {
        error_log($x . " " . $issue['key']);
		// this prints out 1. WH-123, 1. WH-234, etc. for first pass 
                // and 2.WH-123, 2.WH-234 for second pass
		// this is where the problem is coming in; I'm getting the same 50 results for both passes, 
		// instead of the first 50 results in pass 1 and the final 3 in pass 2.
    }
    error_log("----------------------------------------------"); // just to make it easy to see end of each pass
}

curl_close($ch);

Is there anything obviously wrong with what I’m trying to do? Or am I using startAt incorrectly?

Hello @estrom,

I believe I understand what you’re trying to do (although, I am not yet well-versed with PHP), stepping back a bit, were you able to confirm that your $url is always getting the correct value? Based on your code and with a result set of size 53, I assume you would have two different values for $url (one for each pass given $numCalls is 2), I suggest try calling each $url using cURL and compare the results. If your cURL call is similar to the results of your app, then the REST API could be the culprit, otherwise, then something is off with your code.

So far, count($array['issues']) returning zero would explain why the result is like what you’re experiencing.

I tried pagination on my instances, incrementing startAt in every call, and so far the search REST API is working as expected on my end.

I believe you are using it properly.

Cheers,
Ian

Hi @estrom

Two comments / questions :

Are you sure that the %3D is translating correctly to =? Are you able to debug and check this, because if it isn’t correct then it will ignore the startAt and default to 0. Are you explicitly putting in %3D? I don’t know PHP, can you just use =?

Once it’s working, to make it a bit more efficient, you could try to get the number of calls as part of the first run, so if numCalls is 1 you only ever call the API once. Not crucial, but could speed it up a bit.

Warren

Thanks for the reply. Yes, I can confirm that the URL is getting the correct value on each pass. I will try running the hard-coded URLs individually and see what happens.

Yes, %3D is translating correctly.

Hello @estrom,

I am facing the same issue. I see its very old conversation but wanted to check if this issue got resolved. If so, would you please suggest on how to implement startAt in curl command