Expiry of refresh token - Jira Cloud oAuth2 3LO grants

Hey @ekaukonen, our customers are reporting issues with our app that accesses the Jira REST API via OAuth 3LO and we are seeing an increased number of 5xx errors again for requests made to the API via OAuth 3LO. Status codes include 500, 502 and 503 all of which to /ex/jira/{siteId}/rest/api/2/search. We see 500 and 502 the most. 503s are not so common but still happen every now and then.

As suggested before in this thread we have implemented a retry strategy but the fact that we still see these errors regularly suggests that this is not a very effective method to fix these issues. Also, retrying requests that return these errors might just be contributing to the 503 errors (see example below).

Additionally, we still see regular 403 errors. The frequency of these errors makes it hard to believe that they are all customers who have revoked the connection. However, I will try to work with some customers to get some concrete examples for this in a separate post.

Here are some examples for 5xx errors:

500 returns the usual message as reported before by others

{
     error: "Internal Server Error"      
     exception: "com.netflix.zuul.exception.ZuulException"      
     message: "Read timed out"      
     status: 500      
     timestamp: "2019-11-20T09:44:29.781+0000"      
}

Sample response headers in case they contain any useful information for you

{
     connection: "close"      
     content-type: "application/json;charset=UTF-8"      
     date: "Wed, 20 Nov 2019 09:44:29 GMT"      
     strict-transport-security: "max-age=315360000; includeSubDomains; preload"      
     transfer-encoding: "chunked"      
     vary: "Accept-Encoding"      
     x-application-context: "Stargate:prod,prod-east:8080"      
     x-content-type-options: "nosniff"      
     x-failure-category: "SOCKET_TIMEOUT"      
     x-xss-protection: "1; mode=block"      
    }

502: Bad gateway without any extra details

503 returns

This Jira instance is currently under heavy load and is not able to process your request. Try again in a few seconds. If the problem persists, contact Jira support.

We see the message above also sometimes exposed as 429 errors.

Do you have any insights from Atlassian’s side on what could be wrong here?