Marketplace REST API - (Arbitrary?) Rate Limiting since three days

Marketplace General Discussion
, ,
1

Hi,

we run a batch job every night that gets the current license and transaction reports and also retrieves the app pricing (which it then uses to calculate this and that).
This job has been running over two years without any major change - now all of a sudden - we get rate limiting exceptions while retrieving the prices (number of apps x 3 calls) when running in our “nightly” batch time - ~6 am CET (UTC+01:00). When I run the same job when I start working (around 8am) it works fine.
Any insights from the Marketplace Engineering team? Seems like there are some changes going on…

2 Likes
2

Hi @christoffer,
We have noticed some rate-limiting in our logs and are actively looking into this issue. A fix is on its way.

Thanks,
Manju

3

Hi @mbasaralusrinivasa,

thanks for your response. Let me know if I can be of any help.
Today it happened exactly the same way again - rate limited at 5:10:30 AM UTC; but now it was also rate limited at 7:30 AM UTC when I started the job again.

Also if you have a tip how I can retrieve the live pricing without having to fetch every app key and deployment type individually, that would be an area of improvement for our batch job.

Best, Chris

4

Hi @christoffer,
As of 19th Jan, the rate limits have been tuned and this shouldn’t be an issue anymore.

We are looking at making significant improvements to our pricing APIs in the future with the move to the new billing platform. We will keep you posted on these as and when there are new updates.

Would you be willing to try out GraphQL which would allow you to fetch limited data?

2 Likes
5

Hi @mbasaralusrinivasa , thanks for letting me know - I can confirm that for us it has been working since then.

Would you be willing to try out GraphQL which would allow you to fetch limited data?

Sure - let us know when there is something that we can have a look at.
Best, Chris

1 Like
6

Hi @mbasaralusrinivasa - tonight fetching the pricing went down with a 504 Gateway Timeout. It worked when I re-ran it just now.

7

Hi @christoffer thank you for your patience. I dug up the logs and it seems like the 504s were related to the incident we had on the 25th: Atlassian Developer Status - Intermittent Failures while accessing Atlassian Marketplace
The rate limit was probably not the issue here. Do let me know if you see this again. Cheers!

8

Hi, @HarishAnjaneya , thanks for your effort - I can confirm that since I manually reran the batch on Jan 25th it has stayed up.
Cheers, Chris

9

Hi @mbasaralusrinivasa,

the rate limits have been tuned

can you share any informations about this? (e.g. which http attribute hold the informations, etc.)

GraphQL sound interesting, is there anything to play with?

Kind Regards,
Tim