I share the concerns of @rlander and @jechlin in the two posts preceding mine, but to add to this specific comment above:
It will be next to impossible for Atlassian to meaningfully analyze existing usage of APIs. Static analysis of manifests or even entire plugin JARs does not cut it, because among the thousands of apps that will be potentially impacted, you will find plenty of packages that are going to be imported dynamically, accessed through reflection, or otherwise pulled out of thin air. Those are the known issues for the code that you can directly see (within the .JAR), and it does not even scratch the surface of the code you cannot see (think about customer scripting and so on).
It is also not clear to me what would be the purpose of hiding these other Atlassian packages. Vendors already know that these are not part of the official whatever-api package, so we are already using them at our own risk.
We try to make each release compatible versions back as far as we can. Some of our DC apps are compatible back to Jira 8.0. Won’t this break that? If we can’t rely on Jira supplying the correct versions of the libraries then we are going to have to produce multiple versions of our apps that contain different libraries just so that they work (I assume Atlassian would produce a matrix of which versions are required). In reality that isn’t going to happen and it will be the customer that is going to be worst off for this.
This approach seems a lot worse than the existing way. Just change things so that point releases can be made to require testing. So if 9.5.0 is found to have a vulnerability then patch it in 9.5.1 and mark that as needing testing (not auto version support). The vendors will test their version against this and release a compatible version. It doesn’t give the “just Jira needs to release fix” factor but it does away with the bloat and have customers stuck on old versions of apps
To clarify we will continue to export a large number of Atlassian packages(even outside the -api), however, there are places where we are currently exporting what we consider truly internal implementation detail, for example, packages such as .impl., .internal., .spi.*. These packages will often use third-party libraries, in method/class signatures, thus causing said third-party libraries to also be exported to OSGI.
In cases where we propose restriction of a package we will be open to discussion - expected detail to be available Oct/Nov.
Doesn’t really matter at this stage what you “consider truly internal”. It’s too late… you can’t put the genie back in the bottle.
I had a very quick look just what we import from .internal. only, and I get stuff like com.atlassian.event.internal.EventPublisherImpl, com.atlassian.event.internal.LockFreeEventPublisher, loads of com.atlassian.pocketknife.internal.querydsl.*, several com.atlassian.servicedesk.internal.customfields.origin.*.
You will argue that we should not use them, I will argue that your public API is not sufficient for our use case, and our mutual customers will be the losers.
Moreover it’s a pointless argument because I can only check our usages, I can’t check all the millions of ScriptRunner scripts out there.
Is 2024 the year that Atlassian kills on-prem, and this is the Trojan Horse?
We understand there’s a group of apps which rely on multiple internal product APIs and 3rd party libraries. Once we have a complete list of exported libraries we will be open to discussing the potential impact on various apps and possible remediations.
As you noticed the problem of entanglement of 3rd party dependencies in Atlassian API bears the consequence that such 3rd party dependencies need to be available for apps. Our strategy to address this problem focuses on removing usages of 3rd party libraries from Atlassian APIs thus removing this dependency.
We will be introducing this change in the major (platform) release for the products. Major product releases are expected to break backward compatibility and customers are recommended to verify the compatibility of the entire set of installed apps. Customers with a large number of apps might need to wait before all their apps will be compatible with the new product version.
There might be multiple strategies on how to develop and release apps for the non-backwards-compatible product versions. The compatibility matrix which can be defined on Marketplace for every version of the app allows you to separate binaries compatible with older and recent product versions. Mind that this problem might arise with every major product release due to breaking API changes.
Also com.sun.jersey.spi.container.ResourceFilter and some other Jersey 1 classes from this package, but I don’t know if these will be affected as well.
The packages were used as an example of what may need changes - it is not meant to act as a definitive list. We will release a proposed list in Q4 at which stage we invite feedback on the specific items. In places where there are usages of internal packages due to inadequate APIs, we will discuss options going forward.
Due to security pressures and the increasing compliance needs of our customers, we need to start working together on ways that we can make changes. Atlassian is committed to supporting our DC customers for many years to come.
Thanks for the clarifications. I am not an OSGi expert, but I’d like to ask for clarification on this point, because it seems to be a critical one. Is it truly the case that if you export a package with a method signature that references a certain class in a different package, that other class and package is automatically exported? I believe I have seen at least a few individual cases where this was not true. And even if this is somehow the default behavior, is it not something that you can change?
If you meant that the exported packages would not be useful without also exporting those referenced third-party libraries, that is an entirely different story. It is true that some of the methods would be unusable without exporting those third-party libraries, but that’s our problem and not yours. Most of the methods will likely not use third-party libraries in their signatures and I do not see any security-related reason to hide them. Assuming this can actually be done, why throw out the baby with the bath water? Hiding them solely because of a transitive reference creates a ton of work for app developers, and if the third-party libraries can still remain hidden while exporting the package, I do not the security benefit to Atlassian in hiding everything.
As an aside, it seems like Atlassian could also achieve the same security benefit without having to change anything at all in OSGi. It is easy for Atlassian to declare a policy that, from version X.0 of the host product, only the classes in the myhostproduct-api packages are public APIs, and that everything else is “Buyer Beware” and subject to change without notice. Atlassian is then free to upgrade any libraries (third-party or otherwise) whenever it wants. Does that not meet Atlassian’s security goals?
Vendors will be aware of this policy, and as @kbrazulewicz noted above, we must define compatibility matrices for apps that explicitly state which versions are compatible. Vendors are already expected to test and perform needed upgrades to work with newer versions of the host product, and this is exactly what we will continue to do. As far as the goal of ensuring “Marketplace apps remain unaffected by the library upgrades”, isn’t that also met through that process?
@MateuszMiodek When you share the details please also provide information if the platform update includes upgrading to Tomcat 10 and with that a migration from the javax to jakarta packages.
I’m a bit worried that this would likely break all apps compiled against the current platform version, because I think most apps will access servlet requests etc, resulting in the need for separate builds for old and new platform
I’m not sure if a compatibility layer exists (or is even technically possible) that enables apps to run on both platforms with this regard. If you have any information on this I think it would be very useful to the ecosystem.
@MateuszMiodek, am I correct in assuming that the yet-to-be-published list of dependencies that are meant to be provided by the Atlassian platform (i.e. not hidden in the future) has a big overlap with (or even is a superset of) the snapshot-list provided by Atlassian 2.5 years ago in this other CDAC post?
Please mind that Atlassian’s Java API compatibility policy clarifies that major product releases might change the existing APIs and it might not be possible to maintain a single artefact compatible with multiple major product versions. At the same time, we guarantee that apps targeting specific versions will be compatible with future bugfix and minor releases.
I realize that there are a host of valid concerns in this thread. I do think it’s important to note, as someone developing on the DataCenter platform, that I heartily endorse Atlassian doing the hard work of making it easier for them to upgrade dependencies, setting up good boundaries in their APIs, and so on. There are potential sercurity benefits, sure, but deeper than that, this is essential to evolving the Data Center platform. It will undoubtedly mean work for partners & power users. We’re going to gripe about it. Often, we’ll have substantive feedback that you’ll need to take into account.
That doesn’t make it the wrong move.
This is the kind of long term thinking that shows commitment to the DC platform. That’s good! There are plenty of potential pitfalls, but please don’t let the fear & trepidation here scare you off of doing it. Some of us would like to be able to use updated libraries, and so on. Hyrum’s Law is real, but the RFCs, Atlassian’s early access program, & long term support releases ought to give us sufficient wiggle room to weather the storm of API changes.
Here’s to sticking your hand in the beehive! Cheers.
Thank you, Jonny, for your endorsement. It is definitely our priority to invest in the long-term security and stability of the Data Center platform and products. Changes are always disruptive in the short term (also for us). We will do everything we can to assist Marketplace Partners in adopting them.
I think the most critical point about this change for plugin developers is which API Endpoints are no longer available with this change.
I assume there are resources bound on atlassian side during the EAP to resolve issues caused by this change?
It’s very likely that there will remain a few hidden impacts even after we discuss the possible impacts and checking the remaining 3rd Party Libs you will provide. For that case it would be good to be able to resolve these issues during the EAP Phase and not wait for a few month to be able to release a compatible plugin because it depends on something that is no longer accessable
