How to handle slow REST calls in Forge (10 seconds limit)

Hello there,

We have a Forge app live on the Marketplace that scans through some Confluence page content using a CQL search. We are using requestConfluence from @forge/bridge to get pages. We’ve received a support ticket, where the user received a timeout while downloading contents of huge pages in their space. The call took more than 10 seconds and therefore was killed, which broke our app. It looks like there’s nothing we can do at this point (sure, reduce the number of pages we obtain at once but this is also not scalable).
This is the REST call we currently make is:

/wiki/rest/api/search?expand=content.body.atlas_doc_format,content.version&limit=25&cql=space="${spaceKey}" AND (type=page OR type=blogpost)

Does anybody know to handle slow REST calls in forge correctly? What should we do if Confluence takes more than 10 seconds to respond? Since we are calling the endpoint from the Custom UI frontend we did not expect to run into the 10 seconds limit there.
We already received a bad review because of this, but we don’t think there’s anything we can do to fix this on our side.

Thanks for any ideas!

11 Likes

Hi there,

I guess only Atlassian would be able to answer in a meaningful way - but let me try anyway :slight_smile: .
In the documentation of the Forge Quotas, we see the whole overview and a reference to the place where you can discuss the limits you hit and discuss a way forward: https://developer.atlassian.com/platform/forge/platform-quotas-and-limits/#apps-exceeding-quotas-or-limits

The problem that you encounter here is hard to solve, since the limits can be easily hit without any control on your side.
You could try to request the pages without their bodies so the search will return much quicker. With this list, you could parallelize the fetch of the individual pages and process them when they arrive.
This might solve some problems, but in the long run you still might hit the limits of 10 seconds in total - therefore you might want to get in touch with Atlassian.

Have a great day,

Tom

1 Like

Hi @TomMoors1,

Thanks for your response! This thread actually is our attempt to discuss this with Atlassian. :stuck_out_tongue: And since @ibuchanan has liked the original post I’m hoping somewhat that this is already being discussed internally.

I think in our case things like reducing the limit per request wouldn’t really work, because then we would quickly run into another limitation. Namely, the number of forge function invocations. And even if we’re only fetching the content of a single large page, it seems even that request can take more than 10 seconds for certain customers, so that doesn’t really seem to be a solution.

I’m somewhat hoping that Atlassian will reconsider the implementation of requestConfluence from @forge/bridge. Because as far as I can tell the whole problem is that this is invoking a forge function. If it was just using some kind of auth proxy or a similar mechanism instead, that would remove both the 10 seconds runtime problem as well as the number of invocations one. I actually don’t even think merely increasing the runtime limit to say 20 or even 60 seconds would be much of a solution. Sure, it would definitely improve some things, but unless I’m able to work with the Confluence REST API from within my Custom UI iframe like I am from within a Connect iframe (meaning no runtime or number of calls limits), I’m always going to be opposed towards using forge.

Would be happy to hear from someone at Atlassian about this. :slight_smile:

Cheers,
Sven

7 Likes

I’m glad that this topic comes up. We are currently struggeling with the Forge limits as well. With Easy Subtask Templates we’re creating a lot of issues via the “/rest/api/3/issue/bulk” endpoint. We are executing the creation from a backend function. Unfortunately the endpoint is very slow when creating over 10 issues (up to 30 sec.). Our backend function execution quite often gets killed by the 10sec limit which breaks the application. So we ended up, chunking the creation into smaller batches which feels ugly and leads to other problems.

I also agree with @sven.schatter s explanation as we also noticed the described bottleneck of @forge/bridge.

In a first step it would be a great help for our app if Atlassian could increase the max. timeout of Forge backend functions to about 45 seconds (at least for vendors who ask for it) as this would solve most of our customers problems for now.

Cheers
Julian

3 Likes

I believe this is something we’ve been thinking about and working on internally. Let me see if I can’t track down someone who can share more on where are thinking is on this issue.

5 Likes

How long does the endpoints you want to call take without using the bridge? During my forge evaluation I did see that the bridge seems to take at least 2 seconds for any request to the jira REST API (didn’t test confluence yet).
https://ecosystem.atlassian.net/browse/FRGE-232

In case the REST API is much faster than the forge invocation limit it may be better improve the bridge performance (or a faster way to call these Rest APIs without the bridge) than to increase the limit.

3 Likes

Hey @marketa.jancova,

We have recently kicked off a project to change how requestJira and requestConfluence work for custom UI. Once this work is complete these calls will operate from the frontend, improving performance and removing the 10 second limit on fetching results.

8 Likes

Hello @SamSmyth Please, what is the timeline for this new feature or do you have an estimation of when the limit will be removed?

Hey @OnucheIdoko1,

The feature removing these limits will be available within the next 2 months.

1 Like

Thanks guys for all your comments and ideas, I appreciate it. Unfortunately, removing these limits seems to be the only solution for us.

That’s a great new, @SamSmyth! Thank you, this will save our app. :slight_smile:

3 Likes

Hello, will you remove 10 seconds limit only for requestJira and requestConfluence? What about other lambdas?

1 Like

Hey @AlexeyAleinikov,

requestJira and requestConfluence will not longer go through lambda and as such will avoid the 10 second invocation limit. The 10 second invocation limit will still be in place for lambda invocations (resolvers via invoke, etc.).

1 Like

Is there a recommended workaround if the 10sec are not enough?

Thanks for sharing @SamSmyth ! Sounds great!

@FilippoMatraxia unfortunately there’s no workaround for this. You’ll either have to wait until the changes explained by @SamSmyth are implemented so you can do calls from the Custom UI frontend that take longer than 10 seconds. (Note: As far as I understand this will not change anything about the “forge backend” functions!) Or you’ll have to go with Connect instead.

Cheers,
Sven

Hi please, will this limit also be removed when using the UI kit?

Hello, in my plugin I need to scan and process all issue texts and all comment texts within the Jira cloud instance, and there can be a lot of issues and comments in it. How can I do it when the problem of 10 seconds limit of lambda execution exists?

Hi,
Were these changes implemented?
Thanks

Matteo

1 Like

Yes.

27 July 2021

ANNOUNCEMENT Forge API version 2.2.0

The route function can now accept other routes and can be used inside api.fetch(route) . This enables usages like api.fetch(route${baseRoute}/specific/api) .

ANNOUNCEMENT Lower Forge limits

Each app is now limited to 30 logs per invocation. Previously, this limit was 100 per invocation.

ADDED Forge CLI version 1.7.0

This release includes:

  • Added --non-interactive flag to forge install --upgrade command. With the --non-interactive flag, --site , --product and --environment options are required. These flags skip all upgrade prompts and allow install --upgrade to be run in CI.
  • Support for optional chaining in JavaScript files. For example, writing hello?.world in a JavaScript file is now possible.

Run npm install @forge/cli@latest on the command line to install the latest version of @forge/cli .

ADDED Forge bridge version 2.0.0

We’ve released Forge bridge version 2.0.0, which enables custom UI apps to request Jira and Confluence REST APIs directly from the frontend, without the need to proxy them through a custom UI resolver. This means you no longer need to define a custom UI resolver to use the requestJira and requestConfluence bridge methods.

This change also lifts the 10-second timeout previously applied when using resolvers and lets custom UI apps request endpoints that return non-JSON responses, such as attachments.

Run npm install @forge/bridge@latest in your resource directory to install the latest version.

2 Likes

Hi @marketa.jancova !

Have you guys implemented those changes?
It seems to affect deployed forge apps that are using requestConfluence or the storage module I’m using in my Forge app.

There are apps ready to be submitted to your Forge hackathon and this issue just wont go away :expressionless:

A lot of time and effort that went into this, and the developers couldn’t know in advance that once they will deploy their Forge apps they will have these limits.

Again - this issue goes away when you’re tunneling the app instead of deploying it, but there’s no way to tunnel a forge app in production.

Please help!

1 Like

:wave: : Welcome @YogevD. Did you check out the answer above? The changes have been live since July 27.

Can you confirm that you’ve updated to the latest version of bridge?

If you’re still running into problems, I’d recommend opening a new topic.

1 Like