Rate limiting Response handling pseudo code

[OndejMedek]

I am trying to implement rate limiting algorithm by Response handling pseudo code and have some questions about the pseudo code.

1)

If the server responds with status 429 and no Retry-After header, then the delay would be: 10sec, 20sec, 30sec, 30sec because:

• I assume lastRetryDelay is set to initialRetryDelay, so the first delay is 2*5sec = 10sec.
• maxRetryDelay is hit on the 3rd round already, should be initialRetryDelay lower?
• No jitter, because the jitter is computed with retryAfterDelay only, is that right?

Especially the last point baffles me. I would do it other way. I think that retryAfterDelay is authoritative and does not need a jitter while computed delay should have a jitter added and not just min(2 * lastRetryDelay, maxRetryDelay).

2)

No handling of X-RateLimit-Reset, see Retry-After headers. Is that OK? Also, Retry-After may be a <http-date>, see MDN Retry-After. Does Jira API returns Retry-After as a <delay-seconds> only?

3)

The pseudo code does not explicitly computes retryAfterDelay, but is seems it’s assumed in [ms] while Retry-After header should be in [s]. Does Jira API returns Retry-After in [ms] or [s]?

[dmorrow]

Hi @OndejMedek ,

Thanks for posing these questions. I wrote the pseudo code with the aim of trying to keep it generic and simple to understand, but it would be great to see contributions from the community.

Here are my responses to your questions:

1. Yes, lastRetryDelay should be initialised to initialRetryDelay - I’ll fix this. The code is designed to be tuned for different situations so UI operations probably shouldn’t have too many retries, but other non time-sensitive operations perhaps may deserve more retries. The jitter in the pseudo code isn’t clear - I’ll fix it.
2. I think it’s OK not to factor in the value of X-RateLimit-Reset, but maybe doing so would help optimise for certain cases. Happy to hear your thoughts on this.
3. The units of the value returned by the Retry-After header is seconds.

Regards,
Dugald

[OndejMedek]

Thanks for quick answer and clarification. IMHO setting fixed retry time (either in X-RateLimit-Reset or in Retry-After) may be problematic because of clock synchronization problem. So, it’s simpler to avoid that if the Atlassian does not insist on that. (Also the question is, if and how often is this header field used in real retry limit answers.)

[OndejMedek]

@dmorrow What about to use widely used JavaScript open source library ky and document it at the Rate limiting page? If some details or retry algorithm of this library are not aligned with the Atlassian rules, then we may create a feature request to the source repository to improve it.

Update: I’ve found several problems in ky retry algorithm and reported them to the author. It would be nice anyway if we would able to just use some JS library and go.

[dmorrow]

Hi @OndejMedek ,

I just wrote a little NodeJS code to check if my pseudo code makes sense.

const fetch = require('node-fetch');

// See also https://developer.atlassian.com/cloud/jira/platform/rate-limiting

const defaultRetryOptions = {
  maxRetries: 4,
  lastRetryDelayMillis: 5000,
  maxRetryDelayMillis: 30000,
  jitterMultiplierRange: [0.7, 1.3]
}

const apiFetch = {};

/**
 * This method mimics node-fetch, but allows certain options to be passed in to 
 * control how to handle errors.
 * @param {*} url the URL to be called.
 * @param {*} fetchOptions the options as per node-fetch
 * @param {*} retryOptions an object comprising fields maxRetries, lastRetryDelayMillis, 
 * maxRetryDelayMillis and jitterMultiplierRange. See defaultRetryOptions above.
 * @returns the response of the API call.
 */
apiFetch.fetch = async function (url, fetchOptions, retryOptions) {
  const retryCount = 0;
  retryOptions = apiFetch._sanitiseRetryOptions(retryOptions);
  return await apiFetch._fetch(url, fetchOptions, retryOptions, retryCount);
}

apiFetch._fetch = function (url, fetchOptions, retryOptions, retryCount) {
  return new Promise(async (resolve, reject) => {
    try {
      const response = await fetch(url, fetchOptions);
      if (response.ok) {
        resolve(response);
      } else {
        let retryDelayMillis = -1;
        let retryAfter = response.headers.get('Retry-After');
        if (retryAfter) {
          retryDelayMillis = 1000 * parseInt(retryAfter);
          console.log(`* Retry-After header value = ${retryAfter}, retryDelayMillis = ${retryDelayMillis}, retryCount = ${retryCount}`);
        } else if (response.status === 429) {
          retryDelayMillis = Math.min(2 * retryOptions.lastRetryDelayMillis, retryOptions.maxRetryDelayMillis);
          console.log(`* Rate limited without Retry-After header! retryDelayMillis = ${retryDelayMillis}, retryCount = ${retryCount}`);
        }
        if (retryDelayMillis > 0 && retryCount < retryOptions.maxRetries) {
          retryDelayMillis += retryDelayMillis * apiFetch._randomInRange(retryOptions.jitterMultiplierRange[0], retryOptions.jitterMultiplierRange[1]);
          await apiFetch._delay(retryDelayMillis);
          resolve(apiFetch._fetch(url, fetchOptions, retryOptions, retryCount + 1));
        } else {
          console.log(`Giving up and sending response with status ${response.status} (retry count = ${retryCount})...`);
          resolve(response);
        }
      }
    } catch (error) {
      console.log(`Caught error:`, error);
      reject(error);
    }
  });
}

apiFetch._delay = async function (millis) {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve();
    }, millis);
  });
}

apiFetch._sanitiseRetryOptions = function (retryOptions) {
  retryOptions = retryOptions ? retryOptions : {};
  const sanitisedOptions = Object.assign({}, retryOptions, defaultRetryOptions);
  if (sanitisedOptions.jitterMultiplierRange[1] <= sanitisedOptions.jitterMultiplierRange[0]) {
    throw new Error(`jitterMultiplierRange must be an array with the second number beiong larger than the first.`);
  }
  return sanitisedOptions;
}

apiFetch._randomInRange = function (min, max) {
  return Math.random() * (max - min) + min;
}

module.exports = apiFetch;

I gave it a little test and it seems to work, but definitely do additional testing. I hope it helps.

Regards,
Dugald

[YY1]

It seems that I got the similar issue: Response Headers return null

[YY1]

My forge app got this error in the log. May I know if there’s any suggestion to fix it? @dmorrow

BTW, I rewrote the code at the top:

import api, { route } from '@forge/api';

const fetch = (url) => {
  return api.asUser().requestJira(route(url));
}

instead of

const fetch = require('node-fetch')

And also changed to

        const response = await fetch(url);
        if (response.ok) {
            resolve(response);

So that I can call this funtion with:

          response = await apiFetch.fetch(`/rest/agile/1.0/board?type=scrum&orderBy=name`);

[dmorrow]

Hi @YY1 ,

I believe the /rest/agile/{version}/board end point is an expensive operation and therefore more likely to return rate limit responses, but I’m not sure why you’re getting a 500 error - this seems like an error on Jira’s side. Are you able to raise an ECOHELP ticket for this.

Regards,
Dugald

[YY1]

App log from 2rd customer site.

[YY1]

Thanks, I’m not sure about whether it’s my code logic problem or Atlassian Rest API or Site data problem. This is ECOHELP ticket to track my issue:
https://ecosystem.atlassian.net/servicedesk/customer/portal/34/ECOHELP-10844