Log4j upgrade update

Jira Development Jira Data Center Announcements
, ,
5

Update as of Sep 1, 2022

As @RomanKolosovskiy mentioned we are working to upgrade to a new fork of Log4j in Jira and Jira Service Management. Log4j is open-source software originally provided by the Apache Software Foundation that records events – errors and routine system operations – and communicates diagnostic messages about them to system administrators and users.

While Log4j 1 reached the end-of-life state and is no longer maintained by the Apache Software Foundation, Jira Server and Data Center still heavily rely on the library. The amount of effort required to migrate our software to Log4j 2 is too significant for us to be able to do this quickly enough and still maintain a high level of security and performance.

As a solution, we’ve created our own fork of Log4j1 atlassian/log4j1 aiming to resolve security vulnerabilities as they appear and provide the best possible performance while maintaining backward compatibility with the original library.

:warning: Current Jira Core, Jira Software, and Jira Service Management implementations aren’t vulnerable to:

CVE-2021-44228 CVE-2019-17571
FAQ for CVE-2021-44228 CVE-2019-17571 and Deserialization of Untrusted Data
… Some on-premises products use an Atlassian-maintained fork of Log4j 1.2.17, which is not vulnerable to CVE-2021-44228. We have done additional analysis on this fork and confirmed a new but similar vulnerability that can only be exploited by a trusted party. For that reason, Atlassian rates the severity level for on-premises products as low. SocketServer class that is vulnerable to deserialization of untrusted data which can be exploited to remotely execute arbitrary code when combined with a deserialization gadget when listening to untrusted network traffic for log data.

We knew the new version of the Atlassian fork wouldn’t probably reduce the number of problems reported by scanners, but we believe that those reports are false-positive. So we’ve put a lot of effort into maintaining the fork and securing it.

In order to take the extra step to ensure continued compliance, we will upgrade Log4J to >= 2.17.2 within an expedited timeframe which will be released within a to-be-determined feature release version before December 31, 2022.

:warning: This is a breaking change that will require some partners to update their apps to the new version of Log4J in order to remain compatible.

When is it happening?

We provided lates update to the JAC ticket on Aug 6, 2022

The Log4j EAP will start in September.

The Jira/JSM version with the Log4j update will be released in Q4 CY 2022.

What do I need to do?

Recommended approach

We recommend making the logging independent from the product. Don’t rely on the logging backend.

If the plugin uses plain product-defined logging and doesn’t need custom log files or appenders, it should use slf4j-api provided in jira-api.

If access to the full configuration abilities of the logging backends (usually, it’s used to create the plugin’s log files) is required, the best move is to just bundle the logging backend within the plugin.
To achieve this, bundling logging backend (e.g. log4j2, reload4j, or logback) in compile scope is required.

What if I don’t want to/cannot use my own logging backend?

It depends on to what extent has your plugin used Log4j.

To minimise the breaking aspect of the change, we decided to replace log4j in jira-api with the log4j-1.2-api Log4j 1.x Adaptor – Log4j 1.2 Bridge. The real version of this package is 2.x but we are still exporting it to OSGi with a version specified as1.2. Basic logging using this API provided by Jira will still work correctly, but if there was a programmatic log4j 1 configuration in a plugin, it will stop working if the provided log4j was used.

We have also migrated log4j configuration to log4j 2 xml format and our custom appenders (e.g. JiraHomeApender) and layouts to use log4j 2 API.

Scenario-specific migration steps and action required for a Partner/Vendor or App/Plugin Developer

No. What might happen? Migration steps Action required
1 The plugin depends on jira-api and doesn’t have log4j as an explicit dependency or has a provided dependency. Only slf4j API is used in code. Plugins that are even compiled with the old version of Jira should work fine without changes. :warning: TEST WITH EAP
2 The plugin depends on jira-api and doesn’t have log4j as an explicit dependency or has a provided dependency. Only log4j basic logging API and slf4j API is used in code. If only the Log4j API like Logger.getLogger(Something.class).error("error message"); is used, it should work without changes. :warning: TEST WITH EAP
3 log4j1 configuration API is used and/or custom components (like appenders and layouts) are used. All the runtime configurations must be written using log4j2 APIs if log4j provided by the product is to be used. (Log4j – Extending Log4j 2 Configuration) provided scoped log4j-core and/or log4j-api dependencies should be used. org.apache.logging.log4j has to be imported in OSGi if there is no * import already. Log4j1 configuration API will not work correctly when using a log4j-1.2-api bridge. The basic logging API might be left as is. :no_entry_sign: CODE CHANGE
4 Plugin uses Log4j 1 MDC API to store objects. This won’t work with Log4j 2. Preferably, just store the strings using slf4j MDC API. Storing strings will keep working as before. :no_entry_sign: CODE CHANGE
5 Plugin uses Log4j in scope compile, bundled in its jar. It should work as before if the plugin did not rely on log4j from Jira. :warning: TEST WITH EAP
6 Plugin uses com.atlassian.jira.log.LoggingConfiguration This class is removed. You can use your own pattern if you used this class. :no_entry_sign: CODE CHANGE
7 Plugin uses amps log4jProperties property to provide your own Log4j config for testing purposes. Upgrade to 8.7.0 AMPS and >= to 3.6.3 maven, migrate your logging config to log4j 2 xml format. Instead of log4jProperties, log4j2Config property should be used and point it to the migrated configuration file. If you notice problems with WRM.I18n being not initialised or scripts executing in a wrong order, use false in the AMPS config. This is a new AMPS feature that changes behaviour. The property will disable it. :no_entry_sign: CODE CHANGE
8 Plugin uses Log4j APIs to assert on tests. This won’t work with the bridge, Log4j2 API should be used if it’s really required to assert on logs. :no_entry_sign: CODE CHANGE
9 Plugin relies on Jira custom Appender and/or Layout classes (e.g. JiraHomeAppender) This won’t work as intended. We will use log4j 2 configuration. Appender and Layout classes are migrated to log4j2 API. Migration to log4j2 is required. :no_entry_sign: CODE CHANGE

Why does this matter?

Our branch of in-house maintained Log4J-1 is not vulnerable to Log4Shell.

Nevertheless, we want to remove the old Log4J-1 library. The existence of even a well-maintained branch that is not vulnerable doesn’t help our customers navigate this risk internally.

This branched version still registers as a risk for customers in their own scans of our software. This creates a huge amount of friction in their internal processes. We need to do what’s right for our customers. The DHS calls Log4J-1 an “endemic” cyber threat.

What are my resources?

3 Likes