Hi,
We’d like to mark our app Twitter Connector as DC compatible.
The app is free for server and it will remain free for DC as well.
Do we have to run all the performance tests for a free app?
Cheers
Prem
2 Likes
Hey @pch,
If you can explain that there is no probability of impacting the host system due to the architecture of the app. You can skip the perf test with approval on the DCHELP ticket. If there is a possibility of impacting the perf of the host system you need to do the perf tests.
Magro
4 Likes
Very important this use-case is supported. We’re seeing many very tiny plugins go “paid” for datacenter and we suffer death by 1000 cuts on some of these things we’ve built up using Confluence for 11 years.
1 Like
@bmagro To be honest… No, this should not work, as this would break with your own rules. Either there is a required certification or not…
There already was a similar question on Slack some time ago, with the message, that is no shortcut (may @OleksandrMetelytsia remembers).
This would be quite unfair, as many have very simple plugins, with “only macros” and just could “argue” performance issues away, but invested a lot into testing. This also wouldn’t be transparent for customers, as they pay a lot of money and expected a “valid” certification. (at least the “performance test” should be a required minimum)
My personal opinion: the DC certification isn’t well-thought-out.
It will:
- remove free apps, as the certification is just to “expensive”…
- remove small vendors (like us), because we can’t afford the approval in the long term. Already working on it for a long time. Costs a lot of resources (time, money) and we need to gain a lot of new knowledge … the same time Atlassian is removing our source of income (Server)…
- keep away newcomers to join the marketplace, as they can’t develop a basic Server app…
My personal opinion 2: Atlassian wants everyone in the cloud and does everything for this… I just had a look at Atlassian “values”. There still isn’t a point “Don’t #@!% the vendor”
To not only be rude…
My suggestion is to layer the certification
- (Something for free apps, ideas?)
- Certificate: Make sure your app is generally working in DC (“performance test” of dcapt + functional tests without scaling → so “main stream hardware is enough for testing, unit test are sufficient”)
- Certificate: Make sure your app is scaling for many users/content (full dcapt)
my understanding is, there was always the exception for apps with very simple architecture. It does not really matter if its free or not. If your app does not do anything with performance impact or any other negative impact on the host instance, you can argue this with the DC team.
That being said, its very hard to maintain a DC app free or charge and it’s only possible if you have a good income source from other Server apps. I think most free apps will go away on DC and that’s okay. If you want to run a high availability cluster you should not be installing free apps without proper maintenance and support.
2 Likes
If your app does not do anything with performance impact or any other negative impact on the host instance, you can argue this with the DC team.
1.) If there is any app like this … it still should not get the same certification as apps where developers invested heavily into the certification.
2.) How can you “argue” that it doesn’t have any impact?
Even the “simplest” macro may have a huge impact. As soon as an app is implementing a backend feature (“Java at the server”), it should be tested.
Only exceptions coming into my mind are:
- apps without any backend functionality, only relying on existing Confluence REST APIs (i.e. working only with JavaScript/HTML/CSS)
- apps only integrating external apps via iframes
Side note: Before the current certification, the process for DC “certification” was just a questionary and “trust”? Obviously this was a problem.
We have a number of DC approved macro apps that run entirely client-side in the browser. The only server-side code is a license check. There is absolutely no possible chance of our apps impacting server performance.
Yet when I asked the question at Summit/AtlasCamp 2017 (or was it 2018? Travel seems so long ago…), I was explicitly told that ALL apps had to go through the same certification process and there was no shortcut for “simple” apps. (My question can be heard in the video recording of the DC Certified App Program session, if you care to go and find it in the archives)
This would be extremely disappointing if it is possible for new apps to get an exception like this.
I truly hope you misspoke there, @bmagro .
Every year we are required to resubmit performance and scale test results, and if we need to do this then everyone should.
I think the underlying thread here is that the rules can be bend at the discretion of the reviewer, which means that whether or not you have to do the full Monty deepens on who is reviewing you
Hello All,
Thanks again for your comments. just a reminder I have moved into a different team now. @kderenda now looks after this program.
I think there is some confusion here about the definition of a simple app. The distinction I have made in the past is that I don’t care about the complexity of the app. I care about the ability of the app to impact the performance of the system and its ability to scale. If we don’t believe it is possible for the app to make an impact on these things we don’t require them to complete the performance test component of the approval. For every app the details are different that’s why we ask you to engage with the reviewer on the ticket. I’ve personally approved these exceptions in the past. This has nothing to do with new apps vs old apps. The default is these tests are required unless you can make it clear to us why the app can’t impact the performance or scalability of the instance.
I’ve always tried to press with the community that for the DC App program we try to be as human as we can with the process. We are happy to dive into the details of the app to understand complexity rather than blindly following so-called rules. I felt like this has delivered the best outcomes for our customers and we have seen a drastic improvement in app quality since the establishment of the program.
In saying this, I am the first to admit the processes we have in place have never been perfect and we welcome feedback and suggestions on how we can best evolve the program. I’ll make sure these are passed along to @kderenda.
5 Likes
I’m saving a permalink to this thread, so that I can link back to it in our next round of annual reviews.
Now that this precedent has been set, I’m very much looking forward to telling app review “yeah, nah…” when asked to resubmit our latest performance & scale testing results.
Instead of us spending significant time, effort and money standing up a 4-node DC cluster in AWS, populating it with a large data set, and then testing a client-side JS based Confluence macro that can’t possibly have any performance impact; we’ll use that time to work on features.
It really does seems to dilute the value of the whole program for DC customers, but if it saves us time and money, then I’m not complaining.
2 Likes
For what it’s worth……
Our annual review for one of our apps came up a few weeks ago, and we tried to claim an exemption on the basis that our app:
- has NOT changed significantly since our last review
- has been approved for Data Center for a number of years (since inception of the Data Center Approved Apps program)
- is a dynamic content macro that runs entirely in the end-user’s browser AFTER the Confluence page has rendered. The only server-side code that executes is a license check.
Based on the app’s long history participating in the DC Approved Apps program and the architecture of our app, we believe that our app meets the criteria that Ben Magro outlined in this CDAC thread for requesting an exemption to skip the performance tests
The response we got was:
Although your app has not been changed from the previous review, our requirements have. Considering your app introduces UI elements unfortunately it is not possible to skip the test execution for now.
So we ended up having to complete the test process as normal (which we were eventually approved)
2 Likes