I am using Jira Cloud and developing an app for it. When using webtriggers and queues, I am time limited: Maximum 25 seconds a process is allowed to run. But some processes take longer, for example because many users or many issues need to be handled. Using “forge tunnel” this limitation is removed, but I need a solution which works in a traditional way.
The following ideas I have now:
- Use database on own hosters to store the state of the process and to divide the process in several steps, each executed within the 25 seconds limit.
- Execute parts of the process on own hosters => Rest-API binding
- Use of queues. The 25-second limit applies here as well.
- Parallelize processes: For example, using webtrigger and consumer simultaneously. Still, our processes would be so large that we wouldn’t be within the 25-second runtime limit
- Optimize code to optimize execution time. Most of the execution time is taken by the API interfaces within the processes, which we have no control over
- Ask for help from the Atlassian community
- Ask Atlassian to lift the limiter for us: The app should be used in other companies as well, so this would be a cumbersome solution. Maybe there is a way to lift the limitering for our developer apps and the production app or limit it to one hour
- Split processes: But you never know if a single process might take longer than 25 seconds anyway, for example due to a slow connection.
Do you have any suggestions to work around this problem?
think the best option you have available here is too process your task in small batches and process them using forge async events API
Use the retry mechanism provided, it will try upto 4 times, also if you are hitting API rate limits for the API you are calling you might get a “retry-after” header which you can use to retry the Queue after that specified time
thank you for your answer. I already tried it: I still just have 25 seconds for every iteration. I can go up to 4 times, that’s right - but in sum I just won 100 seconds. I need much more time. I could retry every 4 times but it’s also increasing the number of requests in a specific time for each iteration (which is also restricted by atlassian). Also, we need to know which parts of the processes are done already. To communicate with each other, we need to save the data in the first iteration somewhere and get them in the next iteration - and then we have the “too many requests” problem after some time.
@SilasWinter could you explain briefly what is the task you are trying to complete? For example "“creating large number of issues in jira cloud using API” so that I can try to come up with a better solution
Gladly, no problem. I am developing a script to automatically synchronize the worklogs of all employees with an external service. For this we have a button to fetch the time entries since the last synchronization and when they are loaded we confirm the transfer with another button. In the end, several hundred worklogs can accumulate in one day. It is important to mention that we are also dealing with sensitive data here and we are dependent on another service that receives the time entries and processes them further.
We also need to check if the data transfer was successful or if errors occurred. We have no influence on the waiting time from the external service, which could possibly also exceed the 25-second limit.
I see that you are okay with using an external database, have you considered using connect instead of forge?