Announcing Data Center Platform 7.0. Next step to improve our security posture

Announcements
, , , ,
1

Hello, Atlassian Developer Community!

Back in September of 2023, Atlassian set out on a mission to bolster the Data Center platform by enhancing the security of our Data Center products.

Before we go any further, let me introduce the Data Center platform team who oversees a diverse range of cross-product elements serving as the foundation for Data Center products, including Jira, Confluence, Bitbucket, Bamboo, and Crowd. The team’s primary aim is to minimize maintenance costs while offering development tools and recommendations for a more unified Data Center platform experience.

In 2024, our goal is to meet the security standards for vulnerability solutions and ensure that Marketplace apps remain unaffected by the necessary library upgrades required for patching vulnerabilities.

To achieve this, we’ve centralized dependency management. We’re actively working on Data Center Platform 7.0 that has been slated for adoption in major product releases scheduled for 2024.It’s important to note that this Data Center platform release is primarily internal and will be available as part of the product releases. Nevertheless, marketplace partners can access the platform components through repositories like https://packages.atlassian.com/.

What’s new in Data Center Platform 7.0?

The changes in Data Center Platform 7.0 include:

  • Upgrading numerous Atlassian and third-party components so that they benefit from the latest security patches and bug fixes.
  • Eliminating Gray APIs (old and unsupported third-party and cross-product libraries with dependencies) from our Data Center products. This will reduce the scope of third-party libraries and improve dependency management. It will also allow us to enhance our Data Center products more frequently in the future without causing disruptions to your apps or requiring extensive testing and rework. If you wish to keep using these libraries, you’ll need to bundle them in your app .
  • Other notable updates:
    • Revamped Atlassian Rest (we removed Jackson/Jersey and updated JAX-RS to v2)
    • Reduced public JAVA API in Atlassian Plugins, WRM, Web Fragments, and LESS

Component and library changes

We’re removing the following third-party dependencies from Data Center Platform 7.0. If you wish to continue using these dependencies, make sure to include them in your app.

Third-party dependency

com.google.guava:guava

com.google.inject:inject

org.dom4j:dom4j

joda-time:joda-time

commons-digester:commons-digester

opensymphony:oscore

opensymphony:propertyset

opensymphony:sitemesh

com.fasterxml.jackson.core:jackson-annotations

com.sun.jersey

org.tuckey:urlrewritefilter

org.jdom:jdom

log4j:log4j

commons-pool:commons-pool

commons-lang:commons-lang

To enable the removal of the third-party dependencies listed above, we had to ensure they were no longer used in the public API. This means we’ve made changes to the public API of many Atlassian components to decouple them from these third-party dependencies. In some cases, we’ve used the opportunity to consolidate or further restrict the public API where we believe viable alternatives exist. Here’s the full list:

Component Status
net.java.dev.activeobjects:activeobjects-core CHANGED
com.atlassian.activeobjects:activeobjects-plugin CHANGED
com.atlassian.plugins.authentication:atlassian-authentication-plugin REMOVED
com.atlassian.plugins.authentication:atlassian-authentication-plugin-api CHANGED
com.atlassian.config:atlassian-config CHANGED
com.atlassian.core:atlassian-core CHANGED
com.atlassian:atlassian-failure-cache CHANGED
com.atlassian:atlassian-failure-cache-plugin CHANGED
com.atlassian.plugins:atlassian-nav-links-api CHANGED
com.atlassian.plugins:atlassian-nav-links-plugin CHANGED
com.atlassian.plugins:atlassian-plugins-api CHANGED
com.atlassian.plugins:atlassian-plugins-core CHANGED
com.atlassian.plugins:atlassian-plugins-osgi CHANGED
com.atlassian.plugins:atlassian-plugins-schema CHANGED
com.atlassian.plugins.rest REPLACED
com.atlassian.sal:sal-api CHANGED
com.atlassian.sal:sal-core REMOVED
com.atlassian.sal:sal-spi REMOVED
com.atlassian.sal:sal-spring REMOVED
com.atlassian.scheduler:atlassian-scheduler-caesium CHANGED
com.atlassian.scheduler:atlassian-scheduler-quartz2 CHANGED
com.atlassian.security:atlassian-secure-xml CHANGED
com.atlassian.streams:streams-api CHANGED
com.atlassian.streams:streams-spi CHANGED
com.atlassian.streams:streams-thirdparty-api CHANGED
com.atlassian.upm:upm-api CHANGED
com.atlassian.plugins:atlassian-plugins-webfragment-api CHANGED
com.atlassian.plugins:atlassian-plugins-webfragment CHANGED
com.atlassian.plugins:atlassian-plugins-webresource-api CHANGED
com.atlassian.plugins:atlassian-plugins-webresource CHANGED
com.atlassian.plugins:remote-link-aggregator-api CHANGED

How to prepare for Data Center Platform 7.0?

For a more seamless transition, we’re going to deprecate the Gray APIs. Although they’re already labeled as deprecated, they’re still currently accessible in our ongoing product releases (Jira 9.15, Confluence 8.8, Bitbucket 8.18, and Bamboo 9.6). The majority of these will eventually be phased out in our major product releases. Keep an eye out for updates from the product teams regarding their plans for these releases.

If you wish to check the deprecations, removals, and dependencies not available through the product, you can override the versions of the Maven modules listed above to a newer version. In most cases, the latest versions of the modules include the deprecations planned for Data Center Platform 7.0. Over time, we’ll release milestone versions of these modules that include the changes from Data Center Platform 7.0. You can use these milestone versions to estimate the impact on your apps.

What’s next?

We aim to share updates through community posts as we make progress. Feel free to respond to this post with your feedback or questions. In our upcoming update, we’ll delve into specifics on migrating away from the deprecated API and any additional insights we gather along the way.

11 Likes
2

Are platform 7 updates present in the current Confluence 9 EAP?

3 Likes
3

Does the roadmap of these changes also include an update to Atlassian Marketplace to support larger file uploads to accommodate apps bundeling these packages?

14 Likes
4

Is there any details already on JAX-RS v2 todos for App vendors? I really would love if we could avoid these big breaking changes all the time. maybe it would be a good idea again for Atlassian to write their own API layers and hide the actual implementation of lower 3rd party APIs.

Right now it is a lot of work layed upon the app vendors to also constantly adapt to these big changes like the struts one and many others.

But thanks for the advance notice :+1:

2 Likes
5

Hi @rlander - Platform 7 updates are not present in confluence 9 EAP. Keep an eye on the Confluence updates for further details.

1 Like
6

Hi,

Thank @MalathiVangalapati for this post that will allow us to anticipate this important change in our apps.

Would it be possible to add a column in your full list table with the version that contains the changes ?

As you can see in my screenshot, the latest version I found of the activeobjects-plugin is dated January 4, 2024, but I don’t know if this one includes the changes we are currently discussing.

image
image1023×1357 193 KB

We also tried this technique : mvn versions:display-dependency-updates, but based on the results, I doubt it’s usable :no_mouth:

image
image983×312 53.2 KB

Thank you
Fabien [Elements]

2 Likes
7

Hi @remie - The roadmap of these changes do not include an update to marketplace to support larger file uploads. Will add this to the Marketplace backlog and keep this group updated as we make progress.

8

Thank you! Can you please confirm that Atlassian will not ship these changes until this limitation has been addressed by the Atlassian Marketplace team? Because otherwise vendors will not be able to upload versions that are compatible with the new platform release.

Please note that this blocker was already identified and relayed to the DC team by myself and I believe also @jmort or @rlander in the early feedback stages (June 2023).

6 Likes
9

@MalathiVangalapati what are the plans with regard to coordinating this release with Marketplace Partners? Are there going to be EAP’s? What kind of timeframe are we looking at between giving us the ability to test the impact this has on our apps and releasing it?

Because usually customers expect us to have host product compatibility with 2 weeks after the release (and we try to do that as early as possible). But with the possible impact of not only removing OSGi access to these 3rd party packages, but also making changes to Atlassian product APIs, we will probably need a lot more time to be able to support this. And there might be instances in which our apps simply will no longer work because we relied on grey APIs that have no become private / deprecated.

7 Likes
10

I can’t answer all of your questions, but if there is no longer an API that you need, do post please with what exactly the API was, what you were using it for, and the customer use-case. I can’t fix the world, but with that information I can try pass it on to the right teams.

11

My point was more generic and with “we” I meant all Atlassian Marketplace Partners. This isn’t about specific use cases, this is about making sure that as a community we provide the best experience to our mutual customers.

1 Like
12

Hi @MalathiVangalapati,

  1. You implied that platform 7 updates are not currently present in the Confluence 9.0 EAP. Are they expected to be present in a future Confluence 9.0 EAP or is this work targeted for 10.0?

  2. I second @clouless 's request for more information on the JAX-RS v2 updates. Confluence 9 has already withdrawn the Jackson packages from OSGi, so is this part perhaps already shipping despite the comment above? For example, how should vendors replace existing annotations like @JsonProperty and @JsonSerialize? (Both are removed from Confluence 9.) These are fundamental building blocks for REST providers and vendors need guidance on what to replace them with.

5 Likes
13

Hi @remie , our plan is to sort out the size limitation before rolling out the feature. We’d really appreciate some input from the developer community to grasp the sizes you are dealing with as It varies based on the apps and libraries being bundled. Thanks for your help!

14

@MalathiVangalapati what are the plans with regard to coordinating this release with Marketplace Partners? Are there going to be EAP’s? What kind of timeframe are we looking at between giving us the ability to test the impact this has on our apps and releasing it?

EAP details will be provided by individual product teams through the usual release process, and these will include the Platform changes.

Similar to how it was done in the past, for example Jira 9.0 Preparing for Jira 9.0 - multiple EAPs coming your way, the details will be accessible in the change log.

https://developer.atlassian.com/server/jira/platform/changelog/

15

Hi @FabienPenchenat , appreciate you bringing this up. I’ve attached a PDF with all the version details.
Versions with API changes.pdf (94.7 KB)

4 Likes
16

When vendors start to bundle packages removed by Atlassian I expect to see some classloading problems that look similar to this (for example)

Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'void org.apache.http.impl.auth.HttpAuthenticator.<init>(org.apache.commons.logging.Log)' the class loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @7e5c476 of the current class, org/apache/http/impl/nio/client/MainClientExec, and the class loader org.springframework.boot.loader.LaunchedURLClassLoader @7adf9f5f for the method's defining class, org/apache/http/impl/auth/HttpAuthenticator, have different Class objects for the type org/apache/commons/logging/Log used in the signature (org.apache.http.impl.nio.client.MainClientExec is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @7e5c476, parent loader org.springframework.boot.loader.LaunchedURLClassLoader @7adf9f5f; org.apache.http.impl.auth.HttpAuthenticator is in unnamed module of loader org.springframework.boot.loader.LaunchedURLClassLoader @7adf9f5f, parent loader 'app')

I was under the impression that OSGi would isolate class loaders, but that does not seem the case! Is there a way that Atlassian host products can guarantee 100% class loading isolation??

17

Hi @scott.dudley , Yes, the Platform 7 updates are expected to be available in a future Confluence 9.0 EAP. Regarding @clouless and your request, we intend to publish the migration documentation within a few weeks.

1 Like
18

Hi, the issue you are describing happened because:

  1. Class MainClientExec is loaded by a bundle classloader (probably internal dependency of the plugin)
  2. Class HttpAuthenticatoris loaded by a different classloader (judging by name, probably product’s system one - HttpAuthenticator is probably exported by the “System Bundle”)
  3. Both classes use Log class and both loaded it from different sources (probably that class is also included in the plugin)
  4. Class MainClientExec wants to execute constructor of HttpAuthenticator.
  5. Both classes have link to a different copy of Log class that results in a runtime error (similar in effect to Log cannot be cast to Log - common when playing with classloaders)

It is hard to judge what would be the fix in such case, but probably plugin (bundle) should not include org.apache.commons.logging.Log and instead rely on the copy exported to the OSGi. OSGi isolates the loaded classes, but it cannot prevent mixing them at runtime.

Going to the context of our changes coming in Platform 7. Our first goal was to remove 3rd party dependencies from our own API. Only when we assured it is not present there, we marked those libraries to be removed from the OSGi exports. It means you cannot have a linkage error (assuming no bugs :smiley: ), because our API won’t even use those types any more.

To give an example, let’s consider there is a method
com.atlassian.Plugin.getDate() <org.joda.time.LocalDate> that is used somewhere in a plugin. The plugin imports packages com.atlassian, org.joda.time`.

In Platform 7 we replaced that method with com.atlassian.Plugin.getDate() <java.time.LocalDate>. We also no longer export joda-time.

Plugin’s code need to be updated to use the new method. joda-time is no longer used in context related to Atlassian. If that was the only context a plugin need joda-time, the dependency could be removed together with OSGi import.

If joda-time is used in a different context, plugin still can use it. This time it needs to provide the library by itself, as Atlassian API no longer provides it. Plugin local classloader will load those local copies of joda-time classes.

Even if somehow references to those objects will be processed by Atlassian internal code, there won’t be any mismatch. The only possibility it will be processed internally is if it will be used in some other context (e.g. a plugin is registering bean of org.joda.time.LocalDate) and our code will be using a some common interface/super class (e.g. java.lang.Object) to manipulate it. There won’t be any linkage error risk.

I hope this explanation resolved your concerns :slight_smile:

5 Likes
19

Hi @clouless ,

If you feel like doing some spelunking, the epic is public: DCPL-217

1 Like
20

thanks for pointing that out, I got overwhelmed by just looking at the epic :smiley:
I just wait for the docs. Let’s just hope it will be not too much todos for app vendors :smiling_face_with_tear:

1 Like