Hey there,
We’re currently working on a Confluence Cloud app that runs a “time critical” task. With the size of the Confluence instance, the workload of this task increases significantly and we need to heavily parallelize our process so that we don’t go over our time limit.
This of course means that there’s a good degree of complexity involved and we really need to test our code against a big instance before we can go to the Marketplace. And not only do we need this to test our functionality, but we also consider this the only actual way to figure out whether optimizations in our code actually improve performance against the Atlassian REST API.
I remember that at AppWeek 2019 Berlin it was said there was a way to get a large Cloud instance via a DEVHELP ticket. I have such a ticket (DEVHELP-2889) that is now soon to be a year in triage. I’m not expecting to get an answer on it anymore…
In Server / DC land this is not a problem at all and to be honest I don’t understand why Cloud dev instances can only have so few users. This just makes a lot of things unnecessarily hard and complicated. 
So, how does one test a Cloud app against a large data set and/or a large amount of users?
Cheers,
Sven
4 Likes
Hi @sven.schatter,
All Confluence Cloud tenants run on the same hardware stack so the only way that I would imagine influencing the “size” of a tenant would be through adjusting configuration settings that may limit attributes such as the maximum number of allowed users.
Having said that, I’m not familiar with any such settings and have asked around without any luck so far. I’ll post an update if I find anything.
Regards,
Dugald
Hi @dmorrow,
Thanks for taking the time to look into this!
Maybe I should’ve been more clear about that, but what I’m asking for is indeed to get a instance for testing where I can create 1,000 users or even more. I want to be able to put them in different groups. I want to have groups with more than 200 users to assure that pagination works on our end. I want to have groups that can’t access content that others have created… etc. There’s a lot of use cases which can only really be tested with a large set of users and 5 definitely isn’t enough.
Mostly though, this is indeed about performance testing for us. If you need me to go into more detail:
-
We’re gonna have a job that pulls users depending on some configuration. (This could be all users on the instance)
-
Per user we need to fetch the E-Mail address & Display name. (2 REST calls)
-
Per user we’re performing a search and fetching up to 50 items. (1 REST call)
-
We need more information about these items so we need to perform up to two more REST calls per item.
So, we’re ending up with over a hundred REST calls for each user. We have found out that an Atlassian REST call can take anywhere from 100-500ms in our situation.
Now, that means in the worst case (and without parallelization) it could take us more than 50 seconds to finish the work for a single user because everything needs to be a slow REST call. (<CloudRant>On Server all of this is a matter of milliseconds thanks to the Java API</CloudRant>)
We’re running this job on AWS Lambda and as you might know that can only execute for 900 seconds, so we have to split off packets with users into a Queue and distribute them across many different Lambdas. Bla bla bla scalability bla bla you probably see where I’m going with this. 
The thing is, we want to get data on whether what we have done to parallelize our process is enough and how it performs in a real world worst case. We also want to see how this performs in a real world situation because we’re scared that we might get throttled and all that kind of stuff. How does caching certain things affect a real world scenario? This is only possible to test with a real instance of course and we don’t want to find out all of these things in production only.
So yeah, long story short: We need a 1,000(+) User Confluence Cloud instance for testing.
Cheers,
Sven
1 Like