Preparing for Confluence 8.0 - EAP coming soon

Hi @IrinaTiapchenko

thank you for full clarification :slightly_smiling_face:, I will be watching the release notes when this export is officially available.

Cheers
Adam

2 Likes

Hey @jens,

My apologies for the delay in getting back to you on this one, I’ll make sure that doesn’t happen again.

Seems this is deprecated in Confluence 7.5 sources, but not in 7.18 anymore. :face_with_raised_eyebrow:
We’re also defining custom lucene-search-filter-mapper and lucene-boosting-strategy modules.

I’ll follow up with the team that’s working on this on Monday. Unfortunately they’ll be out tomorrow.

Thanks for your patience in the meantime.

Regards,
James Ponting
Engineering Manager - Confluence Data Center

2 Likes

Hey, would someone be able to coordinate with the relevant team to upgrade the latest Plugin SDK so that it supports Java 11?

One cannot compile against the 8.0.0-struts-m020 milestone using JDK8 as noted previously:

bad class file: /.../.m2/repository/com/atlassian/confluence/confluence/8.0.0-struts-m020/confluence-8.0.0-struts-m020.jar(com/atlassian/confluence/pages/AbstractPage.class)
 class file has wrong version 55.0, should be 52.0

The latest plugin SDK 8.2.17 does not support JDK11 and trying to use it produces the following error:

$ atlas-mvn package
Executing: /.../atlassian-plugin-sdk-8.2.7/apache-maven-3.5.4/bin/mvn  -gs /.../atlassian-plugin-sdk-8.2.7/apache-maven-3.5.4/conf/settings.xml package
Exception in thread "main" java.lang.NullPointerException
	at java.base/java.io.Reader.<init>(Reader.java:167)
	at java.base/java.io.InputStreamReader.<init>(InputStreamReader.java:72)
	at org.twdata.maven.trap.Dispatcher.getInterceptorsResourceBufferedReader(Dispatcher.java:67)
	at org.twdata.maven.trap.Dispatcher.getMavenInterceptors(Dispatcher.java:45)
	at org.twdata.maven.trap.Dispatcher.main(Dispatcher.java:21)
	at org.twdata.maven.trap.App.main(App.java:18)

I can build with Conf 8.0 and JDK11 by falling back to a much older version of the plugin SDK (8.0.7), which works for whatever reason. This seems…suboptimal? And the corresponding AMPS-1541 issue to upgrade the plugin SDK is “Won’t Fix”?!

Hi @scott.dudley

please check my posts - Preparing for Confluence 8.0 - EAP coming soon - #2 by adam.labus and Preparing for Confluence 8.0 - EAP coming soon - #4 by adam.labus

I think you are still using Java 8, not Java 11.

Check your configuration in the console using atlas-version, because I have no problem compiling 8.0.0-struts-m020 version on SDK 8.2.7 and Java 11

r2d2@tython restapi-extender_8 % atlas-version

ATLAS Version:    8.2.7
ATLAS Home:       /Applications/Atlassian/atlassian-plugin-sdk
ATLAS Scripts:    /Applications/Atlassian/atlassian-plugin-sdk/bin
ATLAS Maven Home: /Applications/Atlassian/atlassian-plugin-sdk/apache-maven
AMPS Version:     8.1.2
--------
Executing: /Applications/Atlassian/atlassian-plugin-sdk/apache-maven/bin/mvn --version -gs /Applications/Atlassian/atlassian-plugin-sdk/apache-maven/conf/settings.xml
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /Applications/Atlassian/atlassian-plugin-sdk/apache-maven
Java version: 11.0.15, vendor: Azul Systems, Inc., runtime: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home
Default locale: en_PL, platform encoding: UTF-8
OS name: "mac os x", version: "12.5.1", arch: "aarch64", family: "mac"

Cheers
Adam

Hey @adam.labus

Nope, I’m using Java 11. Even atlas-version crashes. Did you patch the version of Maven in the SDK? (Maybe to get ARM64 support?) Plugin SDK 8.2.7 ships with Maven 3.5.4, but it looks like you are using 3.8.6.

$ java -version
openjdk version "11.0.16" 2022-07-19
OpenJDK Runtime Environment Temurin-11.0.16+8 (build 11.0.16+8)
OpenJDK 64-Bit Server VM Temurin-11.0.16+8 (build 11.0.16+8, mixed mode)

$ atlas-version
ATLAS Version:    8.2.7
ATLAS Home:       /.../atlassian-plugin-sdk-8.2.7
ATLAS Scripts:    /.../atlassian-plugin-sdk-8.2.7/bin
ATLAS Maven Home: /.../atlassian-plugin-sdk-8.2.7/apache-maven-3.5.4
AMPS Version:     8.1.2
--------
Executing: /.../atlassian-plugin-sdk-8.2.7/apache-maven-3.5.4/bin/mvn --version -gs /.../atlassian-plugin-sdk-8.2.7/apache-maven-3.5.4/conf/settings.xml
Exception in thread "main" java.lang.NullPointerException
	at java.base/java.io.Reader.<init>(Reader.java:167)
	at java.base/java.io.InputStreamReader.<init>(InputStreamReader.java:72)
	at org.twdata.maven.trap.Dispatcher.getInterceptorsResourceBufferedReader(Dispatcher.java:67)
	at org.twdata.maven.trap.Dispatcher.getMavenInterceptors(Dispatcher.java:45)
	at org.twdata.maven.trap.Dispatcher.main(Dispatcher.java:21)
	at org.twdata.maven.trap.App.main(App.java:18)

Hi @scott.dudley

in fact, after downgrading Maven to 3.5.4 build fails.

My configuration which allows me to compile the app for 8.0.0-struts-m020:

  • Maven - 3.8.6
  • SDK - 8.2.7
  • AMPS - 8.6.0 - pom.xml = <amps.version>8.6.0</amps.version> - i don’t know why atlas-version shows 8.1.2
  • Java - 11

Cheers
Adam

@adam.labus Yeah, thanks, I got it working with Java 11 after upgrading Maven manually. It’s unfortunate though that one can’t even build against the latest Confluence release using the stock plugin SDK, without jumping through these undocumented hoops.

See also:

2 Likes

Hey @jens,

Thanks for your patience on this one.

Some of our apps define lucene-query-mapper modules. Am I understanding the announcements correctly, that this will no longer be possible (because the mapper implementations create Lucene objects)?

Seems this is deprecated in Confluence 7.5 sources, but not in 7.18 anymore. :face_with_raised_eyebrow:

The mapper in question is no longer marked as deprecated as we’re using it in Search V2’s implementation. That said, this mapper and the classes within org.apache.lucene will be sealed off in Confluence 8.0. So functionally the same thing.

With no direct access to the Lucene API, custom mappers can no longer be used. Instead a custom SearchQuery can be used in place, however you’ll need to override the expand() method that decomposes itself into multiple primitive SearchQuery classes such as BooleanQuery, TermQuery etc.

We’re also defining custom lucene-search-filter-mapper and lucene-boosting-strategy modules.

Will there be any replacement APIs for 3rd party apps? This would result in a major refactoring of our apps and most probably a loss of features otherwise.

We removed lucene-search-filter-mapper module mainly as org.apache.lucene.search.Filter has been replaced with a Query class in Lucene 6 (see Apache Lucene Migration Guide). In this case, you’ll need to replace SearchFilter you’re using with a new SearchQuery and, again, will need to override the provided expand() method.

The lucene-boosting-strategy module was deprecated in Confluence 7.0, so you’ll need to replace its usage with the new FunctionScoreQuery, in addition to any score functions defined under com.atlassian.confluence.search.v2.score.

Hopefully this gives you what you need :slight_smile:

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

2 Likes

What are the plans for app developers who want to keep their apps compatible for both Confluence 7.x and 8.x in parallel? For example, according to the Struts2 upgrade, packages from com.opensymphony.xwork.* should be changed to
com.opensymphony.xwork2.*. It would be extremely costly to have to publish the same apps for Confluence 7.x and 8.x differently.

1 Like

Hi @MatthiasClasen

please check this answer → PSA - Confluence moves to Struts in 8.0 - EAP - #6 by ggautam

Cheers
Adam

3 Likes

Dear @Wendy and all,

based on what I read here in the linked documents I did not expect any changes to the v2.ContentSearch. However we are now seeing the following exception:

java.lang.NoSuchMethodError: ‘void com.atlassian.confluence.search.v2.ContentSearch.(com.atlassian.confluence.search.v2.SearchQuery, com.atlassian.confluence.search.v2.SearchSort, com.atlassian.confluence.search.v2.SearchFilter, int, int)’

Do you have any additional information on that?

Thank you and best regards

Andreas

Update 2022-09-21: I assume the problem comes from the removed SearchFilter classes; they will be deprecated in 7.20 and removed in 8.0; the new ContentSearch is here

2 Likes

Hi @andreas1,
That constructor method (and any method signature that has com.atlassian.confluence.search.v2.SearchFilter) is deprecated in 7.20 and will be removed from 8.0 as mentioned in the javadoc (that you have already linked to).

@jponting @ttranminh @ggautam (sorry, I don’t know who to ping here)

As of today the EAP downloads page only provides access to m69, whereas Preparing for Confluence 8.0 does still list m60 as the latest EAP release.

I downloaded m69 and upgraded my existing m45 installation by replacing the distribution files. After startup it told me that some of my apps where disabled, so I tried to uninstall them.

This resulted in an NullpointerException in some Xwork class:

java.lang.NullPointerException: null
	at com.opensymphony.xwork2.config.ConfigurationUtil.buildParentsFromString(ConfigurationUtil.java:53) ~[org.apache.struts_struts2-core-2.5.30-atlassian-3.jar:2.5.30-atlassian-3]
	at com.atlassian.confluence.plugin.descriptor.XWorkModuleDescriptor.buildPackageContext(XWorkModuleDescriptor.java:339) ~[com.atlassian.confluence_confluence-8.0.0-m69.jar:?]
	at com.atlassian.confluence.plugin.descriptor.XWorkModuleDescriptor.getPackages(XWorkModuleDescriptor.java:118) ~[com.atlassian.confluence_confluence-8.0.0-m69.jar:?]
	at com.atlassian.confluence.plugin.descriptor.XWorkModuleDescriptor.destroy(XWorkModuleDescriptor.java:380) ~[com.atlassian.confluence_confluence-8.0.0-m69.jar:?]
	at com.atlassian.plugin.manager.DefaultPluginManager.notifyUninstallPlugin(DefaultPluginManager.java:1087) ~[com.atlassian.plugins_atlassian-plugins-core-7.1.4.jar:?]

Attaching a debugger revealed that the XWorkModuleDescriptor instance from the stack had set its configuration field set to null, resulting in the NPE.

This exception apparently interrupts the app uninstallation process in such a way that I’m unable to remove the app.

Just wanted to let you know.

Not sure if it’s a temporary thing, but I suppose the EAP downloads page should provide access to the milestones you consider stable enough for mentioning/linking on the ‘preparing for’ page.
In the meantime m60 can be downloaded from the maven repo.

1 Like

I downloaded m69 and upgraded my existing m45 installation by replacing the distribution files. After startup it told me that some of my apps where disabled, so I tried to uninstall them.

This resulted in an NullpointerException in some Xwork class:

Thanks for bringing this to our attention @jens. This exception is indeed an edge case we had not handled. We’ve created a card to track this and related XWorkModuleDescriptor issues here.

2 Likes

Hi Jens, I wanted to share that we updated the Preparing for 8.0 page with the latest m69 EAP milestone when it was made available to download on our EAP downloads page. However, we must’ve experienced a caching issue at the time. We’ve reset it now so hopefully, everything matches up and there’s no additional confusion. As always, thank you for raising this. :slight_smile:

1 Like

Hey @Kusal, I’m not sure if it’s really just an edge case.

I removed all user-installed apps from my m60 installation, then upgraded to m69. No errors during startup.

Then I installed one of our apps. The app installation failed because this app still OSGi-imported the old com.opensymphony.webwork package, which is not available in m69 anymore.
Now I also cannot uninstall that app anymore. It is again failing with the NPE I mentioned above.

After removing the plugin from the plugindata table and restarting Confluence I’ve been able to install a fixed version, but as you can imagine it’s not very convenient to do this, especially while testing app compatibility which usually involves a couple of reinstalls.

What’s more confusing: Afterwards I installed another app, which also failed due to com.opensymphony.webwork, but then I could uninstall that one just fine. :man_shrugging:

In any case it would be awesome if you could fix this in the next milestone or share some ideas how we could avoid this if you know more about it.

Hi @ggautam, milestone 69 broke one one of our apps.

The app has a servlet that expects multipart/form-data via POST, but now whenever the servlet is invoked, the request body is empty.
I assume this is because there’s now a ConfluenceBootstrapAwareStrutsPrepareFilter (subclass of StrutsPrepareFilter) in the call stack that wraps the request in a org.apache.struts2.dispatcher.multipart.MultiPartRequestWrapper which seems to consume the request’s input stream.
The filter seems to be mapped in com.atlassian.confluence.impl.webapp.ServletFilters:

    // Add mapping for dispatcher type to handle error page from web.xml section to handle servlet url /trigger500Error
    // then SITEMESH_ERROR will handle all url
    STRUTS2_PREPARE(filter("struts2-prepare", new ConfluenceBootstrapAwareStrutsPrepareFilter())
            .mapping(EnumSet.of(REQUEST, ERROR), ALL_URLS)),

This is pretty bad because our app needs to process the request body itself. How can we opt out of this?

Hi @Kusal and @ggautam

I share Jens’s concern about the uninstallation problem (CONFSERVER-80124), which is currently marked as low priority. Can this please be escalated, fixed and released quickly? This is a blocker for a reasonable dev loop for any third-party developer.

@jens - it sounds like your use case is that you need to get at the multipart data stream directly. On the off chance you get forced to use the multipart wrapper anyway because of architectural limitations, or for others who come across this thread, it looks like this wrapper might sometimes also be nested to an arbitrary degree. I note the existence of a helper method in com.atlassian.xwork.FileUploadUtils#unwrapMultiPartRequest to safely grab a MultiPartRequestWrapper from an HttpServletRequest.

2 Likes

Now I also cannot uninstall that app anymore. It is again failing with the NPE I mentioned above.

Thank you for the additional feedback @jens and @scott.dudley. The fix for this will be included in the next milestone and will remedy all uninstallation scenarios that result in the aforementioned NPE/stacktrace.

2 Likes

The app has a servlet that expects multipart/form-data via POST, but now whenever the servlet is invoked, the request body is empty.

This is pretty bad because our app needs to process the request body itself. How can we opt out of this?

Hi @jens, thanks for bringing this to our attention. I’ve created a card for this which you may track here.

1 Like