hi @rlander. Atlassian Security Scanner is based on Snyk, but has extra features on top of it, e.g. skipping of provided dependencies.
Dependency management and security scanning of third-party dependencies is the responsibility of the app developer and you could use any tool you like.
You could request your app scanning via DCHELP ticket, but Security Scanner API is not publicly available at the moment.
We already raised a feature request to Snyk team to filter out by scope, but it is not implemented yet. So we made our own filtering from .pom files based on scope.
Again, you could use any tool you like for keeping your third-party dependencies up to date. Atlassian Security Scanner is an extra check from the Atlassian side, but not a tool that we are forcing everyone to use.
Any provided dependency could be ignored from the app partner side.
There are internal tickets raised for dev teams to fix vulnerabilities in provided Atlassian modules. So at some moment, the latest provided dependencies from Atlassian will be clean from third-party vulnerabilities.
I reviewed the documentation links provided and I see no mention of the concept of whether a third party dependency is either upgradable aka fixable, only its severity score. Can you please address this?
for example this is what we use in some of our pipelines. the EXTRA_ARGS “–fail-on=upgradable” is key in gating decision
App developers should scan for vulnerabilities as required per the new Data Center App Approval program policies. They are highly encouraged to embed this scanning in their CI pipelines.
The app should be free from critical- and high-severity security vulnerabilities in third party dependencies, so if discovered they should be addressed by the app developer prior to receiving DC approval.
Unless such vulnerability is discovered in an Atlassian provided dependency, because obviously Atlassian does not have to abide to the same set of rules as Marketplace Partners, because the goal that customers can trust our software to be safe and secure does not actually apply to Atlassian.
App developers can use different scanners, but the recommended scanner is Snyk.
Oh right, Snyk actually does not really work, as it will give heaps of false positives because it does not exclude provided dependencies. App developers are expected to be able to parse these results themselves in order to decide which are Atlassian provided (and as such exempted, because Atlassian) and which should actually be fixed prior to receiving DC approval. You can also open a DCHELP ticket to have us scan your dependencies with our proprietary scanner based on your dependency tree.
Oh yeah, that’s right, Atlassian has actually created an alternative scanner based on Snyk which will filter out these vulnerable provided dependencies (which, again, we’re not going to fix properly but just ignore) so that we can quickly check if the app complies for DC approval requirements. Unfortunately we will not make this scanner public, because no reason.
This program is consistent with existing internal development practices. Atlassian has been focusing heavily on security and working hard behind the scenes to resolve vulnerabilities. This is an ongoing commitment and we take it seriously. At times we find resolution to a vulnerability at odds with the promise not to break API between major releases, here we conduct further analysis and risk assessment. Where there are mitigations available this results in us making the decision not to upgrade a vulnerability until a later date.
We will be adding more examples of how to run different tools in the next few days. Some of these do support suppression of provided so we hope you find them satisfactory alternatives.
In terms of making our scanner available publicly this is not the focus of this program, we wish to encourage good security practices amongst DC App vendors rather than be prescriptive on exact tools used. Our ultimate goal is to make our products more safe and secure for our customers.
Obviously, any vendor can decide on whatever tooling they prefer to use to satisfy your requirements. But I don’t see how Atlassian sharing the tools they already have would take away from that.
You literally have the ability to make this entire process less painful for every vendor, saving us tons of time, in which we can focus on the things that you purport to care about: safer and more secure products.
But instead, you are intentionally making it more difficult for us, asking us to reinvent solutions to problems you already solved, spending tons of useless hours in the process.
I fail to see how this lines up with you saying that the goal is to make things more safe and secure.
No seriously: help me understand how your answer, and your decision to withhold tooling, reconciles with your stated goal.
To be clear, I fully support the idea of scanning DC apps for vulnerable dependencies, and this is something I would expect vendors to be doing anyway. Having Atlassian push for this is a good thing for everybody, as long as it is done with care.
My team already use OWASP Dependency-Check to scan for vulnerable dependencies, and we have it configured to filter out runtime/provided dependencies.
I would not expect Atlassian to mandate a specific tool. However, all apps will be required to be scanned by your security scanner as part of the DC review. By having each vendor implement their own variation of scanning without good guidance, you increase the risk of discrepancies in results from the various tools/configurations, which could lead to wasted time for vendors.
If Atlassian are going to scan my app with a tool (the results of which could impact our annual DC review), then I want to be proactive and run the exact same scan for every build of our app, be that on a local developer machine or CI.
At the very least, I would like to see a tutorial/instructions on how to configure a dependency scan for an app built with the Atlassian SDK, including filtering out noisy runtime dependencies. Nobody should be mandated to use this exact tooling, but as the majority of the ecosystem uses the SDK (we don’t FWIW), it would be a good starting point to avoid each vendor having to do analysis from the ground up.
The current instructions provided for using Snyk to run a scan are not fit for purpose, evidenced by the feedback of this thread, and the fact Atlassian had to build internal customised tooling for the reasons raised by the vendor community here.
Hi @rlander. Thank you for your feedback. I totally agree with your first paragraph. We are open to feedback and what to make the approval process useful for everyone.
And that is great that you are using OWASP Dependency-Check to scan for vulnerable dependencies.
Atlassian scanner is not able to handle the whole variety of the edge cases giving the fact that all apps are unique and have their own development processes. Think of the variety of packaging tools, multi-language modules, private dependencies, custom approaches of package management, jars with no information about dependencies at all, etc and etc.
We do not want to force anyone to use any specific tooling for scanning or for packaging the app, instead, we want to introduce a sanity check from the Atlassian side for those who never thought that outdated third-party dependencies could potentially lead to serious security implications.
That is a fair point that our example reference instructions are not good, we are working on improving them and we will update the documentation in a near future.
You are not forcing anyone to use anything. By making your scanner available to the Atlassian Marketplace Partners, you allow us to properly prepare for the DC approval process, as it is to be expected that the results of your scanner will trigger discussion on the DCHELP ticket. To avoid friction in this proces, it would really help if Marketplace Partners are looking at the same scanner results.
There is plenty of tooling that Atlassian has made available and that we are not forced to use. For instance, the Performance Testing Toolkit is not mandatory for partners to use. We are technically allowed to use other means to provide you with a similar result (CSV files). Nor is anyone required to use the AWS templates or (new) Kubernetes / Helm charts for deploying Data Center. But we will be using them because it lowers friction during the process.
So if the Marketplace Partner community is giving you the feedback that your argument of not wanting to force anything upon us is not something we deeply care about, and rather have access to the Scanner, would you reconsider? Or are there other arguments that you have not yet shared with us?
Thank you for your response @OleksandrMetelytsia, I am glad to hear you will be making some documentation updates.
I understand the point about Atlassian’s scanning being a “sanity check” to make sure vendors have not missed any issues with their own tooling, although I do think there should be some basic guidelines for how to setup such tooling, e.g with the Atlassian SDK.
We would still like to be able to run Atlassian’s sanity check ourselves in our CI pipeline, we would not expect it to be perfect, we just want to be able to compare output between our tooling and Atlassian’s as early as possible. Having to raise a ticket to compare things is a bit too much friction.
Is it the case that you don’t want to make the scanner available, because it has a lot of limitations, which could lead vendors into a false sense of security, vs a more comprehensive scan via some other tool operating at the source code level? I could empathise with that point of view.
Yes, @rlander you’ve summarised everything correctly. There are a lot of tools and approaches for more in-depth scanning at the source code level than our generic scanner which is looking at third-party dependencies only. And we do not want to create a dangerous impression that if the Atlassian scanner did not find anything in third-party dependencies, then the app is security vulnerabilities-free.
I do understand why you want to run the same “sanity” check in your CI, but at the moment we do not have yet a good solution for this. So for now you could ask in your DCHELP for Atlassian scan results for example a month before your annual review date. We will post updates when we come up with a better solution for this.
Atlassian scanner is not able to handle the whole variety of the edge cases giving the fact that all apps are unique and have their own development processes. Think of the variety of packaging tools, multi-language modules, private dependencies, custom approaches of package management, jars with no information about dependencies at all, etc and etc.
We do not want to force anyone to use any specific tooling for scanning or for packaging the app, instead, we want to introduce a “sanity” check from the Atlassian side for those who never thought that outdated third-party dependencies could potentially lead to serious security implications.
Forgive me, but there is some weird logic in here. Although Marketplace Partners should always perform in-depth vulnerability scans, this is not something you can enforce. However, you are adding that requirement to the DC approval process, because you do want to enforce it, and you will be enforcing it with a sanity check using a proprietary tool, however to prevent Marketplace Partners from only relying on that tool you are not making it available. Except Marketplace Partners are not required to run any tool, they only have to pass the sanity check
May I propose adjusting the DC approval requirement to include A) an output of any vulnerability scanner and B) the output of mvn dependency:tree (or similar in other build tools)? That way you get to know that we are running a in-depth vulnerability scanner (which is what you are after) and allow the DCHELP process to verify it with your private sanity check.
Because if you only expect us to pass the sanity check (if that is the DC requirement) you should make that sanity check available to us.
@remie B) this is exactly what we propose with this announcement.
And A) I would love to see this in the DCHELP ticket, but my impression was that it would be an extra step for the app partner. Right now a lot of apps on the marketplace have 0 critical+high vulns in third-party dependencies.
If there are no objections from other partners I would love to add A) into the DC requirements.
By adding the requirement of running a vulnerability scanner, at least it will remove any unclarity about what you are trying to achieve here and will probably help Marketplace Partners find ~90% of issues that will be required to fix prior to submitting their DCHELP ticket (or annual review).
Even though I’m not really looking forward to implementing this, I would say that adding the requirement of providing with in-depth scanning results instead of only mvn dependency:tree at least clarifies the intention of the DC approval program and I think it will remove a lot of friction on the DCHELP ticket.